Baidu created Unlimited OCR, an open-source extension of DeepSeek OCR that is able to process up to 40 pages in a single forward pass. It’s like having an AI OCR model that can read an entire book in one go.
The model’s architecture is inspired by human working memory during book copying. Rather than memorizing everything that has already been written, a person repeatedly looks back at the original document while keeping only a limited amount of recently written text in mind. Similarly, Unlimited OCR continuously attends to the visual representation of the original document and a fixed-size sliding window of recently generated tokens to predict the next output token (see the picture below).
This approach enabled Unlimited OCR to process documents of up to approximately 40 pages under a 32K context length, according to the authors’ experiments.
Limitations of modern OCR systems
As modern OCR systems are now AI-powered, they are able to recognize complex document layouts such as tables, multi-column pages, forms, invoices, charts, mathematical equations, handwritten notes, and mixed text-image documents. However, they have an important limitation when processing long documents. As they generate more text, the decoder’s Key-Value (KV) cache continuously grows, increasing GPU memory consumption and slowing down inference. As a result, processing large PDFs, such as books, lengthy contracts, technical manuals, or extensive document collections, becomes progressively less efficient.
To solve this problem, Unlimited OCR introduces Reference Sliding Window Attention (R-SWA), a new attention mechanism that keeps the KV cache at a constant size, no matter how long the document is (see the picture below).
The grid shows the attention map, where the rows correspond to newly generated tokens and the columns correspond to context tokens they focus on. The blue squares are the original visual tokens extracted from the OCR input image (the reference tokens). The pink squares represent the previously generated output tokens that the next token focuses on (the working memory tokens).
Each new token pays attention to all the visual tokens from the image and only the most recent output tokens, keeping the memory size (KV cache) fixed throughout the whole decoding process.
The model’s architecture
Unlimited OCR builds on the previously released DeepSeek-OCR, whose architecture is shown in the following figure.
The new model retains the DeepSeek-OCR architecture’s DeepEncoder, which combines SAM-ViT and CLIP-ViT and compresses visual tokens by 16×. For example, a 1024×1024 image is reduced from 4,096 to 256 visual tokens, significantly lowering the visual context the decoder must process while preserving information relevant to OCR.
The main modification is the decoder. Instead of standard Multi-Head Attention (MHA), Unlimited OCR uses Reference Sliding Window Attention (R-SWA). During decoding, each token attends to all visual reference tokens but only a fixed window of recent text tokens. This reduces attention computation and keeps the Key-Value (KV) cache at a constant size rather than allowing it to grow with the document length.
Together, these changes enable efficient long-form document processing with substantially lower memory requirements, making Unlimited OCR practical even on hardware with limited GPU memory.
How Reference Sliding Window Attention (R-SWA) works
R-SWA works by mimicking how a human transcribes a long document. When typing out a physical page, you don’t keep re-reading every single word you typed 10 pages ago. You keep the whole physical document in front of you to see what needs to be transcribed next, while keeping only your recent spot in the text in your immediate short-term memory.
R-SWA translates this real-world human behavior into a mathematical two-stream attention mechanism:
- The reference tokens are fixed: It gives the model a permanent anchor to the source document (reference tokens). The reference tokens are made of the visual tokens (the image) and the prompt. This data is processed once at the very beginning. Once loaded into the KV cache, it never grows. If the image and prompt take up 1,024 tokens, that block of the cache stays exactly 1,024 tokens forever.
- The generated output attention is kept to size n (128 by default): This is the “Sliding Window” part. Instead of letting the model look back at all previously generated text tokens, they limit the output attention to only the preceding n tokens (where n=128). When you generate token 129, you drop the KV cache for token 1. When you generate token 130, you drop the KV cache for token 2.
Total KV cache size = reference tokens (fixed) + n (fixed)
By keeping the sliding window at 128, the model can see the entire image without letting the text history consume too much RAM.
Comparison with existing OCR systems
The table below highlights the main differences between traditional OCR systems, DeepSeek-OCR, and Baidu’s Unlimited OCR. Traditional OCR focuses primarily on text recognition, whereas the two vision-language models provide end-to-end document understanding by recognising text while also interpreting document structure and layout. Unlimited OCR extends these capabilities by introducing multi-page inference, allowing several pages to be processed together in a single inference. This approach preserves context across page boundaries, making it particularly effective for long and complex documents. In addition, its Reference Sliding Window Attention (R-SWA) maintains a constant-size Key-Value (KV) cache during decoding, improving memory efficiency compared with conventional attention mechanisms.
| Feature | Traditional OCR | DeepSeek OCR | Unlimited OCR |
| End-to-end document understanding | Limited | Yes | Yes |
| Long-document support | Limited | Moderate | Excellent |
| Constant KV cache | No | No | Yes |
| Multi-page inference | Limited | Moderate | Excellent |
| Open source | Varies | Yes | Yes |
| Local deployment | Often limited | Yes | Yes |
Overall, Unlimited OCR is better suited for long-document processing while remaining open source and suitable for local deployment.
Training
The team constructed approximately 2 million document OCR data samples to train Unlimited OCR, with a 9:1 ratio of single-page to multi-page data. Unlimited OCR was trained for 4,000 steps with a global batch size of 256 and a maximum sequence length of 32K on 8×16 A800 GPUs, using random packing for all data. During training, they froze the DeepEncoder and only trained the LLM parameters.
Why freeze the visual encoder? Since the DeepEncoder was already optimized during baseline pre-training, freezing its parameters significantly reduced the computational overhead. This allowed the training resources to be used entirely for optimizing the LLM decoder for the R-SWA mechanism.
Evaluation results
Unlimited OCR was evaluated mainly on OmniDocBench v1.5 and v1.6, as well as on long-document parsing tasks. Unlike traditional OCR benchmarks that primarily measure character or word recognition, the OmniDocBench benchmark assesses a model’s ability to understand entire documents, including complex layouts, tables, formulas, and multilingual content.
On OmniDocBench v1.5, Unlimited OCR achieved an overall score of 93.23, compared with 87.01 for the original DeepSeek-OCR (see the next picture). This is an improvement of 6.22 percentage points. You can find the category scores and the model references in the paper.
On the newer OmniDocBench v1.6, Unlimited OCR achieved an overall score of 93.92, which the paper reports as a new end-to-end state-of-the-art result (see the picture below).
On overall score, Unlimited OCR outperforms the other compared models, such as DeepSeek-OCR 2, Qwen3-VL, and OCRVerse. However, independent evaluations across different datasets and production environments will provide a more complete picture of its strengths and limitations.
How to use Unlimited OCR
Getting started with Baidu’s Unlimited OCR takes four main steps:
- Install DependenciesSet up a Python environment and install PyTorch, Hugging Face transformers, and PDF image utilities.
- Download and load the model weights directly from Hugging Face.
- Prepare documents: convert PDF pages or long documents into image files (e.g., PNG/JPEG) before running inference.
- Run OCR Inference: pass single images or all document pages together into the model to extract structured Markdown.
For more details, check the GitHub repository.
Conclusion
Baidu’s Unlimited OCR, a 3B Mixture-of-Experts (MoE) vision-language model built on the DeepSeek-OCR baseline, introduces a new paradigm for long-document OCR. Built on an efficient architecture that activates only about 500M parameters per token, it uses a novel Reference Sliding Window Attention mechanism to keep GPU memory usage constant during generation. Instead of analyzing pages one by one and losing structural continuity, it reads up to 40 pages in a single step to deliver unified, highly accurate Markdown outputs for complex multi-page files.
Read more:
- Paper on arXiv: “Unlimited OCR Works”
- GitHub repository
- Hugging Face Space demo
