模型部署

1、DeepSeek-OCR 提供了 Docker 镜像，用于快速启动 vLLM 推理服务。可使用以下命令启动服务：

sudo docker pull m.daocloud.io/docker.io/vllm/vllm-openai:nightly
sudo docker tag 4fdf harbor-cmp.zoomlion.com/library/vllm/vllm-openai:v0.11.1-preview

2、执行命令

vllm serve /data/modelRepository/ocr/ocr_deepseek-ocr --served-model-name deepseek-ocr --no-enable-prefix-caching --mm-processor-cache-gb 0 --host 0.0.0.0 --port 8080 --chat-template /data/modelRepository/ocr/ocr_deepseek-ocr/template_deepseek_ocr.jinja --api-key fe21b93dd0234e64a8ab44d4c49cf365

未开：--logits_processors vllm.model_executor.models.deepseek_ocr.NGramPerReqLogitsProcessor

~~python3 -m vllm.entrypoints.openai.api_server /data/modelRepository/ocr/ocr_deepseek-ocr --served-model-name deepseek-ocr --logits_processors vllm.model_executor.models.deepseek_ocr.NGramPerReqLogitsProcessor --no-enable-prefix-caching --mm-processor-cache-gb 0 --host 0.0.0.0 --port 8080~~

3、测试脚本

import time
from openai import OpenAI

client = OpenAI(
    api_key="fe21b93dd0234e64a8ab44d4c49cf365",
    base_url="<http://deepseek-ocr-tc-aigctest.toolchain-test.zvalley.com/v1>",
    timeout=3600
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image_url",
                "image_url": {
                    "url": "<https://qmcrm.zoomlion.com/cmm-nginx-beta/document/file/v1/unsigned/download?fileId=574ac51a54b142bc912ed62a480efca0>"
                }
            },
            {
                "type": "text",
                "text": "Free OCR."
            }
        ]
    }
]

start = time.time()
response = client.chat.completions.create(
    model="deepseek-ocr",
    messages=messages,
    temperature=0.0,
    max_tokens=500,
    # ngram logit processor args
    extra_body={
        "vllm_xargs": {
            "ngram_size": 30,
            "window_size": 90,
            # "whitelist_token_ids": [128821, 128822],
        },
        "skip_special_tokens": False,  # whitelist: <td>, </td>
    }
)
print(f"Response costs: {time.time() - start:.2f}s")
print(f"Generated text: {response.choices[0].message.content}")

官方部署DeepSeek-OCR

1、拉取镜像：

FROM continuumio/miniconda3:latest

# 创建 Python 3.12.9 环境
RUN conda create -n deepseek-ocr python=3.12.9 -y

# 把环境写到 bashrc，容器启动后自动激活 deepseek-ocr
RUN echo "source activate deepseek-ocr" >> ~/.bashrc
ENV PATH /opt/conda/envs/deepseek-ocr/bin:$PATH
ENV CONDA_DEFAULT_ENV deepseek-ocr
COPY . .
RUN pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url <https://download.pytorch.org/whl/cu118>
# RUN wget <https://github.com/vllm-project/vllm/releases/download/v0.8.5/vllm-0.8.5-cp38-abi3-manylinux1_x86_64.whl> && pip install ./vllm-0.8.5-cp38-abi3-manylinux1_x86_64.whl
RUN pip install vllm-0.8.5-cp38-abi3-manylinux1_x86_64.whl

RUN pip install -r requirements.txt
# RUN wget <https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.3/flash_attn-2.7.3+cu11torch2.6cxx11abiFALSE-cp312-cp312-linux_x86_64.whl>
RUN pip install flash_attn-2.7.3+cu11torch2.6cxx11abiFALSE-cp312-cp312-linux_x86_64.whl

2、构建镜像：docker build -t deepseek-ocr:v20251023 -f Dockerfile .

3、执行镜像

image.png