56 scanned pages to a cited answer in 5 minutes: RAG pipeline for the documents search cannot read

Upload a scanned 56-page plant guide, and ask which trees tolerate shallow limestone soil. In less than five minutes, you’ll have an answer, along with the page that it came from and the applicable plants that you can buy. The retrieval augmented generation (RAG) pipeline behind it has two managed pieces. jina-ocr-v1 reads each scanned page and returns structured Markdown with its tables intact, and Elasticsearch’s semantic_text field chunks and embeds that Markdown at ingest. As a result, you don’t need chunking code or an embedding job. Hybrid retrieval fuses keyword and semantic rankings before Claude Haiku 4.5 writes the grounded answer, and all of it runs on Elastic Inference Service (EIS) inside one project. This matters because most of the knowledge that organizations rely on never makes it into a database. It lives in PDFs or scanned forms, or in photographed documents that traditional search cannot read. Classic optical character recognition (OCR) pipelines got at it by chaining separate models for layout detection, text recognition, table parsing, and reading order, each with its own cost and failure modes. The new generation of vision-language document parsers does the same job in a single model pass. When paired with a vector-capable search engine, a PDF, or a photo of one, it becomes a queryable, citable knowledge source in minutes. Combining document parsing with hybrid search and RAG lets teams: Make tables, handwriting, and complex layouts searchable, where keyword search and legacy OCR miss them. Ask natural language questions against documents and get grounded answers with page-level citations. Combine exact term matching (such as SKUs or botanical names) with semantic matching (meaning and intent) in a single query. Keep the entire pipeline, including parsing and embeddings, along with generation, inside one platform with one bill and a single security boundary. In this post, we walk through adding a document-parsing feature to an ecommerce site for Texas native plants. (I love plants!) You upload the guide, watch jina-ocr-v1 parse every page, index the output into an Elasticsearch index named after the document, and query it through a RAG chatbot that cites sources by page number and recommends products from the catalog. The stack is as follows: Jina OCR (jina-ocr-v1): A document-parsing model that reads a page image and returns clean, structured Markdown in a single multimodal pass. EIS: GPU-accelerated inference included with Elastic Cloud Serverless. EIS hosts jina-ocr-v1 behind a preconfigured chat_completion endpoint, alongside embedding models and conversational large language models (LLMs) (in this build, Claude Haiku 4.5), so there’s no model deployment or GPU provisioning, and there’s no separate vendor account. EIS usage is metered per token and billed as a Serverless add-on. Current rates by model are on the Elastic Cloud pricing table. Elasticsearch: Stores the parsed documents in the semantic_text field type, which chunks and embeds text at ingest through an inference endpoint and serves hybrid retrieval that fuses lexical and semantic rankings with reciprocal rank fusion (RRF). React, TypeScript, and Vite with an Express back end: The storefront, extended with a document upload page that streams parsing progress over server-sent events, and a floating chat widget. The Express layer keeps all credentials server side. How is jina-ocr-v1 different from classic OCR pipelines? jina-ocr-v1 is a single model; that is, layout, text recognition, table reconstruction, and reading order happen in one pass. Traditional OCR pipelines did the same work by chaining separate models for layout detection, text recognition, table parsing, and reading order reconstruction. Its output is built for retrieval: Structured Markdown with reading order preserved and section headings kept as Markdown headings. Tables emitted as HTML with their values intact, so a 50-state license schedule or a plant care grid remains a table. Handwriting transcribed inline. Repeated page furniture (including running headers and footers, along with printed page numbers) dropped, so it never pollutes retrieval; page numbers come from the ingestion loop instead. That clean structure is what makes chunking, embedding, and citation trustworthy downstream. And it reads one page image per pass (legibility holds up to about 2048 x 2048 pixels), so someone has to run the page loop. Where the page loop runs: Jina Reader API, chat completions, or EIS Who runs that loop is the architecture decision. Route Who runs the page loop Input granularity Where it runs Jina Reader API (r.jina.ai) Jina, server side One PDF per call Outside your Elastic project; separate keys and billing Raw chat completions API Your application One page per call Outside your Elastic project EIS (.jina-ocr-v1-chat_completion) Your application One page image per call Inside your Elastic project; one key, one bill This post uses the EIS route; that is, one platform boundary, and the page loop is a few dozen lines in server/documents.js. Why a scanned plant guide is a hard test for document parsing A demo corpus is only convincing if it’s challenging. Our test document is Native and Adapted Landscape Plants: An Earthwise Guide for Central Texas, an educational publication codeveloped by the City of Austin and the Texas AgriLife Extension Service to help the public with water conservation and eco-friendly gardening. It’s the plant list behind the city's Grow Green program. Its 56 scanned pages are exactly the type of content that basic PDF text extraction gets wrong. Nearly every page is a dense multicolumn table packed with abbreviations (such as, ″VL″ for very low water, and ″B/E″ for Blackland Prairie and Edwards Plateau), check marks for wildlife value and deer resistance, botanical names in italics, and photo captions interleaved with data. A parser that reconstructs those tables with their values intact can handle your invoices. The plant theme also gives document intelligence a home. The host application is a Texas native plant storefront with a seeded Elasticsearch catalog: 48 plants with semantic_text descriptions and a full-text versus hybrid search toggle. Add document intelligence, and a shopper can upload a regional planting guide and then ask Which trees tolerate shallow limestone soil? They’ll get an answer grounded in the guide, with pages cited and a list of products that fit. Prerequisites: Elastic Cloud, an API key, and a scanned document An Elastic Cloud account (a free trial works). Create an Elastic Cloud Serverless project. The vector database project type fits this embedding-heavy workload. In your project, confirm that the EIS model list includes Jina OCR v1 (.jina-ocr-v1-chat_completion) and Claude Haiku 4.5 (.anthropic-claude-4.5-haiku-chat_completion). Both are preconfigured, so there’s nothing to deploy. An Elasticsearch API key with index read/write and inference privileges, and your project endpoint URL. Node.js 20 or later, npm, and git. A test document, such as the Native and Adapted Landscape Plants guide (linked above), or any scanned, table-heavy document of your own. Set up the RAG pipeline for scanned PDFs Step 1: Clone the repository, and install dependencies. git clone https://github.com/elastic/elastic-labs.git cd elastic-labs/supporting-blog-content/jina-ocr-v1-hybrid-rag-elasticsearch npm installStep 2: Configure credentials. Copy the template, and fill in your project values. The .env file is gitignored, so secrets never reach the repository. The OCR endpoint is preconfigured and the embedding model is the project default, so the chat model is the only model choice in the build, and it’s already made for you. cp .env.example .envELASTICSEARCH_URL=https://<your-project>.es.<region>.elastic.cloud:443 ELASTICSEARCH_API_KEY=<your-api-key> LLM_INFERENCE_ID=.anthropic-claude-4.5-haiku-chat_completionStep 3: Seed the product catalog. The seed script creates the catalog index (long_description is a text field mirrored into semantic_text, so hybrid product search works immediately) and bulk loads 48 plants. The first run takes about a minute because Elasticsearch embeds every description at ingest. npm run seedStep 4: Start the back end and the front end in separate terminals, with the back end first. In the first terminal, start the Express API. It listens on port 3001 (set by PORT in .env) and runs the OCR pipeline and chat. npm run server In the second terminal, start the Vite dev server for the React app. It proxies /api calls to the back end. npm run dev Vite prints the URL that it’s serving, normally http://localhost:5173. (It picks the next free port, if 5173 is taken.) Open that URL; don’t browse to port 3001 directly, because it only exposes the API. How the OCR pipeline turns scanned pages into Markdown The Documents page is where the parsing story unfolds on screen, rather than in a terminal. A user drops in a PDF or photo, and the Express server runs a four-stage pipeline while streaming every step back to the browser over server-sent events. Stage 1: Render PDF pages to images PDFs are rendered to PNG page images in memory at roughly 1224 x 1584 pixels, under the model's 2048-pixel legibility ceiling. A photographed document skips straight through as a single page. Stage 2: Transcribe each page, and rebuild its tables Each page image is sent to the EIS endpoint as a base64 data URL, exactly like a vision call to any chat model. Four pages run in parallel, and order is preserved on reassembly: const resp = await fetch( `${ES_URL}/_inference/chat_completion/.jina-ocr-v1-chat_completion/_stream`, { method: 'POST', headers: { Authorization: `ApiKey ${ES_API_KEY}` }, body: JSON.stringify({ messages: [{ role: 'user', content: [ { type: 'image_url', image_url: { url: `data:image/png;base64,${png}` } }, { type: 'text', text: 'Transcr