kreuzberg-dev/kreuzberg
 Watch   
 Star   
 Fork   
1 days ago
kreuzberg

v1.2.1

Fixed

  • (swift): externally tagged enums now decode the wire the core types actually emit. Regenerated on alef 0.87.1. EntityCategory, PiiCategory and ConfidenceSemantics relied on Swift's synthesized Codable, which keys every variant ({"person":{}}), while serde writes a bare string for a fieldless variant and a single-keyed object whose value is the payload ({"custom":"foo"}). Every bridge-constructed value carrying one of these threw DecodingError.typeMismatchEntity and PiiEntity decode their category into a non-optional field, so any result containing an entity failed. OutputFormat is handled separately: its Custom(String) variant carries #[serde(untagged)], so it round-trips as a bare string rather than a keyed object.

Changed

  • Dependencies upgraded across the workspace, including crawlberg 1.6.1 → 1.6.3 and tree-sitter-language-pack 1.19.0 → 1.19.1. skrifa stays pinned at 0.46: 0.47 moves to read-fonts 0.44 while harfrust 0.13.3 is still on 0.43, and the PDF text rasterizer passes a harfrust FontRef to skrifa's OutlineFace.
1 days ago
kreuzberg

v1.2.1

Fixed

  • (swift): externally tagged enums now decode the wire the core types actually emit. Regenerated on alef 0.87.1. EntityCategory, PiiCategory and ConfidenceSemantics relied on Swift's synthesized Codable, which keys every variant ({"person":{}}), while serde writes a bare string for a fieldless variant and a single-keyed object whose value is the payload ({"custom":"foo"}). Every bridge-constructed value carrying one of these threw DecodingError.typeMismatchEntity and PiiEntity decode their category into a non-optional field, so any result containing an entity failed. OutputFormat is handled separately: its Custom(String) variant carries #[serde(untagged)], so it round-trips as a bare string rather than a keyed object.

Changed

  • Dependencies upgraded across the workspace, including crawlberg 1.6.1 → 1.6.3 and tree-sitter-language-pack 1.19.0 → 1.19.1. skrifa stays pinned at 0.46: 0.47 moves to read-fonts 0.44 while harfrust 0.13.3 is still on 0.43, and the PDF text rasterizer passes a harfrust FontRef to skrifa's OutlineFace.
1 days ago
kreuzberg

v1.2.0

Added

  • ServerConfig gained job_timeout_secs (default 600 seconds, override via XBERG_JOB_TIMEOUT_SECS or server.job_timeout_secs) as the configurable fallback timeout for POST /extract-async jobs whose request does not pin down extraction_timeout_secs. A per-request extraction_timeout_secs still always overrides it, and an explicit extraction_timeout_secs: null still falls back to this server cap rather than running unbounded. Previously this fallback was a hardcoded 300 seconds, inconsistent with the 600 second default used everywhere else. See the Changed section for the source-compatibility impact.

  • A long-running process can now release the embedding and reranker models it no longer uses. embeddings::evict_model and reranking::evict_model (and the same functions in sparse_embeddings and late_interaction) drop one model, clear_engine_cache drops every model in a cache, and xberg::clear_engine_caches drops all of them. set_engine_cache_limit bounds the number of resident engines in a cache and drops the least recently used one first. The default stays unbounded, so existing callers see no change (GH#1626).

Fixed

  • OutputFormat.custom in the Python binding always returned None, even when the value genuinely was a custom format. The accessor derived a discriminator from the enum's JSON, which works for every tagged representation but not for this variant, whose payload is untagged and so carries no discriminator to find. It now matches on the variant directly and returns the label.

  • The Python type stub declared a type: str attribute on 23 enum classes that have no such attribute at runtime. The stub emitted it for every data enum, while the runtime only exposes it for enums carrying an explicit serde tag -- so for externally tagged enums (EntityCategory, PiiCategory, OutputFormat) a type checker accepted category.type, which raised AttributeError on use. Stub-only change; no runtime behaviour moved.

  • The Ruby binding's FormatMetadata.from_hash and DiffLine.from_hash read a _0 key that no longer exists on the wire, so every variant deserialized with a nil payload. All 24 call sites are corrected: the 21 FormatMetadata variants now build their payload from the flattened hash, and the 3 DiffLine variants read the text key the enum's #[serde(content = "text")] actually emits (#1594).

  • The Python type stub declared format-metadata payloads as _0 (for example _0: ExcelMetadata), a key present neither on the wire nor on the runtime object -- the runtime per-variant getters such as .excel were always correct. The stub now names the real fields, so type checkers stop reporting valid code as an error. Runtime behaviour is unchanged (#1594).

  • The Elixir Xberg.FormatMetadata typespec documented a metadata: payload key that matched neither the NIF struct nor the serialized wire. It now matches the struct the NIF actually returns (#1594).

  • A Type0 (composite) font's content-stream character codes are now translated to CIDs before glyph widths and vertical metrics are looked up, instead of being used as if they already were CIDs. The two coincide only for Identity-H/Identity-V, which is presumably why this went unnoticed. A PDF using a non-Identity predefined CMap (UniCNS-UCS2-H, UniJIS-UCS2-H, UniGB-UCS2-H, UniKS-UCS2-H, or their -V/UTF16 counterparts) or an embedded /Encoding CMap stream got a wrong width for nearly every glyph -- some over-advancing by up to 4x through the /DW fallback, others under-advancing -- which rendered as stretched or overlapping text and, in extracted text, could split one sentence into a spurious extra paragraph. CIDs now resolve from an embedded /Encoding CMap stream's own begincidrange/begincidchar data when present (including a variable-width codespace), else from the font's /CIDSystemInfo character collection for the four Unicode-keyed predefined families above, else via Identity-H/Identity-V as before. Measured over the 230-document local PDF corpus: 227 byte-identical -- the expected result, since most PDFs use Identity-H -- and 2 changed, both merging text that a stale glyph position had fragmented. Legacy multi-byte predefined CMaps this crate carries no code-to-CID table for (90ms-RKSJ-H, GBK-EUC-H, B5-H, the UTF8 family, UniJIS-UCS2-HW-*, UniJISPro-*, and others) are unchanged -- still wrong, not newly broken -- and now log once per font instead of failing silently (GH#1631).

  • A reconstructed PDF table cell now reads left to right instead of in the order its words happened to arrive. The reported symptom was a sub/superscript printing after the rest of the cell -- eta_S % came out as eta % S, Q_HE GJ as Q GJ HE -- because a script is drawn as its own content-stream segment a fraction of a point below the line it annotates, so every reading-order sort upstream placed it after the whole line. The same defect also transposed values between columns when two columns were merged into one cell: on a balance sheet whose header reads 2017 2016, the row beneath it emitted the 2016 figure first, silently attributing each year's number to the other year. A cell's words are now grouped into visual lines and ordered left to right within each line. Grouping first is load-bearing -- ordering by horizontal position alone interleaves the two halves of a wrapped cell. Each word's own whitespace is also collapsed, so a segment carrying a trailing space no longer stacks it on the separator. Table cells recovered by OCR go through the same ordering (GH#1628).

  • Image OCR now honours a PNG's embedded pHYs pixel density instead of assuming 72 DPI. A genuine 300-DPI PNG submitted with target_dpi = 300 was resized anyway, because the extractor decoded, resized and re-encoded the image -- discarding the density chunk -- before the OCR backend, and therefore before the existing ocr.backend_options["source_dpi"] override, ever saw it. Embedded density is now resolved at the extractor boundary and at the backend from one shared implementation, with the explicit override still taking precedence over it. An image carrying no density metadata still defaults to 72 DPI and still resizes (GH#1630).

  • A body paragraph is no longer deleted for repeating text that appears elsewhere on the same page. The second strip_repeating_text pass keyed on lowercased paragraph text with no check that a table was involved, so a sentence matching an earlier title -- differing only in case, with no table on the page at all -- was silently removed. The pass now runs only on pages that have a detected table, removes a paragraph only when that table's own cells carry the same text, and compares case-sensitively. Measured over 230 PDFs: 44 documents changed, 3472 words recovered and 32 lost, both loss cases inspected and benign (one is a restructure whose total content grew, the other two mojibake tokens) (GH#1623).

  • OCR text is no longer discarded when a scanned page region is detected as a table but its cell grid cannot be recognised. recognize_single_table returned nothing whenever TATR failed, produced no rows or columns, or the grid failed validation, which threw away every OCR element that had been assigned to that region. A region that cannot be recognised as a table now falls back to emitting its text in reading order, and only when that text is not already carried by one of the page's paragraphs, so nothing is duplicated. Together with the restructuring-heuristic retention guard below, recognised OCR text is no longer silently lost on the layout path (GH#1622).

  • extraction_confidence no longer reports a failed structured extraction as fully schema-valid. The pipeline passed SchemaCompliance::AllValid unconditionally, which is 40% of the combined score under the default weights, so a run whose LLM call failed -- or that was built without the liter-llm feature, or ran on wasm -- scored exactly as high as one that validated. A requested structured_extraction that leaves no structured_output now scores AllInvalid. Extractions with no structured_extraction configured are unaffected and keep their previous score; ConfidenceSignals is unchanged in shape, so no serialized form moves (GH#1624).

  • detect_mime_type_from_bytes no longer refuses text that is not valid UTF-8. A byte buffer with no filename or declared type -- a Windows-1252 or ISO-8859-1 CSV export, say -- returned UnsupportedFormat even though the extractors that would receive it decode legacy encodings through encoding_rs. Such content is now reported as text/plain, the same answer the UTF-8 path already gave for the same document, so the two encodings of one file behave alike. Content holding a NUL byte, or with too few printable bytes to read as prose, is still rejected (GH#1625).

  • The Go binding no longer discards the message of every error the native layer reports. Each known error code was mapped to a typed sentinel (ErrTimeout, ErrParsing, ErrOcr, and ~20 more) and returned before the message was ever read, so the detail the native layer had already produced -- observed durations, limits, plugin names, counts -- was dropped for all of them; only unrecognised codes kept their text. A timeout surfaced as the sentinel's own placeholder-stripped text, extraction timed out after ms (limit: ms), which reads as a formatting bug but is the whole message the binding ever had, and left callers unable to tell which timeout had fired. The message is now read first and returned alongside the sentinel, so errors.Is(err, xberg.ErrTimeout) still matches while err.Error() carries the real interpolated text. Go was the only binding affected; C#, Java and Zig already read the message before switching on the code. Regression in 1.1.0, when the typed sentinels were introduced.

  • The Python package's public option classes regained from_json. from xberg import ExtractInput resolves to a generated dataclass that shadows the native class at the same name, and that dataclass carried none of the native class's methods, so ExtractInput.from_json(...) raised AttributeError while xberg._xberg.ExtractInput.from_json(...) worked -- the same name meaning two different things depending on the import. 134 public classes were affected. The dataclasses now delegate from_json to the native class, so both import paths behave the same. Other native-only methods on those classes (validate, is_empty, the PaddleOcrConfig.with_* builders) are still absent from the dataclass twins and are tracked separately.

  • An extraction cancelled by extraction_timeout_secs now actually stops its per-page PDF OCR work. The timeout fires cancel_token.cancel() at every timeout site, but nothing in the OCR page fan-out read the token, so pages kept being OCR'd after the caller already had its Timeout error -- burning CPU and holding OCR concurrency permits, which degrades later extractions in a long-lived process (a server, or anything extracting in a loop). The token is now checked both before spawning a page and inside each spawned task, because the spawn loop finishes almost immediately while tasks queue on the OCR semaphore long after it. A cancelled run also reports Cancelled instead of tripping the all-pages-failed guard and reporting a wholesale OCR backend failure.

  • PDF no longer promotes ordinary body text to a heading. Two gates decide headings independently and neither tested the line's shape, so any line past the title-length floor could be promoted. The sentence-boundary check that should have caught this looked for a literal ". " followed by a capital, but paragraph text joins a block's physical lines with a newline, so every sentence boundary landing at a line end was invisible to the gate while the renderer joined the same lines with a space and displayed it -- the gate and the output disagreed about what the text was. Boundaries are now found across any whitespace, and a line that is mostly bare numerals is treated as a flattened data row rather than a heading. Across 490 documents: 484 unchanged, 5 with fewer headings, 0 with more (GH#1599).

  • Hardened the document-global heading/list heuristic's safety check on the scanned-PDF layout-markdown path (use_layout_for_markdown / layout detection, force-OCR route). That heuristic rebuilds paragraphs from bare line geometry with no knowledge of the ML layout regions the OCR path already classified, and can silently drop a line its own font-clustering pass treats as furniture or noise; the guard against this only checked that the whole document still had one non-empty element, so a single surviving word anywhere passed it even if an entire page's body vanished. The guard is now a per-restructuring canonical-character retention check against the lossless OCR assembly, and falls back to that lossless assembly whenever any content would otherwise be lost. Compares characters rather than word tokens: a restructuring pass legitimately re-wraps text across the line boundaries it reads (measured case: "list of findings" split across a line came back "list offindings", one dropped space), and a word-token comparison read that benign re-wrap as content loss and rejected legitimate heading/list promotion along with it. This closes a real gap in the guard's own logic; it was not reproduced against a specific "entire page lost" report and should not be read as a confirmed fix for one (GH#1622).

  • PDF table/paragraph assembly (assemble_page_elements_with_tables) now suppresses a paragraph whose words a positioned table's own grid fully carries, so a recognized table no longer also renders its flattened source text as an ordinary paragraph immediately next to the grid -- observed directly (not inferred) on a scanned-PDF fixture with layout detection enabled, where a table's status-row text appeared once as prose and once as a correctly gridded table. Mirrors the GH#1616 precedent from the other direction: suppression requires the table's own cell/markdown content to actually account for every one of the paragraph's words (an order-insensitive multiset match, since a reconstructed grid can reassemble the same words in a different order than the source paragraph), not geometry alone, so a paragraph carrying text the grid does not represent still survives. Note: this closes the duplication for paragraph/table pairs that share a coordinate space (the native-PDF table path). Investigating this also surfaced a separate, unresolved coordinate-space mismatch between OCR/TATR-recognized table bounding boxes and OCR paragraph bounding boxes on the force-OCR + layout-detection route specifically, which currently prevents this same guard from geometrically matching on that route; fixing that is out of scope here and is not yet done (GH#1622).

  • PDF no longer deletes text a table's bounding box covers but its grid leaves out. Suppression of text a table already renders was decided on geometry alone, and a reconstructed grid need not span every printed column inside its own bounding box. On a four-column fault-finding grid reconstructed with two columns, every run in the two omitted columns vanished from the document — not in a cell, not in any element, nowhere. A covered run is now suppressed only when the table actually carries its text (GH#1616).

  • PDF no longer cuts a numbered heading that wraps onto a second line. The wrap exemption compared the two lines' right edges, and a wrap's last line is short by definition, so it could never fire: the heading kept only its first line and the rest of its title was emitted as body text. A heading's own continuation is now recognised by its left edge, which is the title's hanging indent rather than the margin body text returns to. Regression in 1.1.5 (GH#1615).

  • An extraction that never requested OCR no longer fails when no OCR backend is registered. ocr-pipeline can be enabled without any backend — ocr implies ocr-pipeline, not the reverse — and in that build the automatic scanned-page trigger aborted an ordinary PDF extraction with OCR backend 'tesseract' not registered. Automatic triggers now check availability and skip with a warning; an explicit force_ocr, force_ocr_pages, ocr_inline_images or caller-supplied ocr config still fails loudly (GH#1610).

  • Legacy binary .ppt now reports which slide each embedded picture belongs to. Pictures were read from the OLE Pictures stream, which stores blips in save order and names no slide, so every extracted image carried no page number and every image node was emitted after the last slide. A slide whose only content is a picture therefore produced nothing at all on its own number and read as a blank slide, and captions or any other data keyed on an image's page were filed against the end of the deck. The owning slide is now resolved through the drawing that references the blip; a picture no live shape references is still extracted, without a slide (GH#1620).

  • Legacy binary .ppt no longer extracts deleted slide revisions or presents slides in the wrong order. The format is append-only across saves, so editing a deck leaves superseded copies in the stream; treating every Slide container as a slide produced 190 slides for a 96-slide presentation, numbered by byte order. Live slides and their order now come from the persist chain (Current UserUserEditAtomPersistDirectoryAtom) and the document's slide list, falling back to the previous behaviour if the chain cannot be read in full. Slide numbers are the page every element and chunk of a deck is cited by, so both defects reached consumers as wrong page numbers (GH#1614).

  • PDF de-hyphenation no longer welds a compound whose own hyphen falls on a line break. Two sites decide whether a trailing hyphen survives; only one consulted the lexical evidence, so long-term, cost-effective and antigen-presenting came out as longterm, costeffective and antigenpresenting — tokens that do not exist, and so unreachable by any lexical search. The assembly site now asks the same question the paragraph site already asked, weighing both the static compound list and the witnesses collected from the document itself. A hyphen the wrap genuinely inserted is still removed (GH#1613).

  • Legacy binary .ppt no longer loses slide titles. PowerPoint keeps a slide's text in two places, and the extractor read only one: titles held in the document-level outline collection (SlideListWithText) landed in the loose-text bucket, which is discarded whenever any slide exists, so they were absent from the output entirely. Outline text is now attributed to its slide by persist order and merged in, skipping any line the slide's own drawing already carries so a title drawn on the canvas is not duplicated (GH#1612).

  • The documented install versions for Java, Kotlin Android, Swift, Zig and the spring-ai integration no longer lag the release. These snippets sit outside task version:sync, which covers the generated API-reference badges but not hand-authored install directives, so they had been telling users to install 1.1.3 (GH#1593 covers the same class of staleness in test_apps, which is still open).

  • OcrConfig no longer rejects valid Tesseract language codes such as fao (Faroese) with Invalid language code 'fao'. Use ISO 639-1 or ISO 639-3 codes.. Config validation checked the language against a general-purpose allowlist that was missing 66 codes Tesseract actually supports, while a separate, Tesseract-specific list already carried them; the two lists had never been reconciled. Config validation itself only started running for configs loaded from files, JSON overrides, or set programmatically in 1.1.0 (previously it ran only in tests), which is when this allowlist gap first became user-visible. Both validators now read from one shared list of Tesseract-supported codes, so this class of divergence cannot recur (GH#1621).

Changed

  • Breaking (Java binding): enum constants now follow Java's own convention and are SCREAMING_SNAKE_CASE instead of carrying Rust's PascalCase verbatim -- LinkStyle.Inline becomes LinkStyle.INLINE, across roughly 75 generated enums. The JSON wire value is unchanged; only the Java identifier moves, so serialized documents and stored payloads are unaffected. Update references to the constants themselves; generated default values (ChunkType.Unknown, OutputFormat.Plain) moved with the declarations.

  • Breaking (Ruby binding): an externally tagged enum variant carrying a single payload (EntityCategory::Custom, PiiCategory::Custom, OutputFormat::Custom) now serializes the way the Rust core always did -- {"custom" => "my-label"} -- instead of wrapping the payload in an extra object keyed by a synthesized positional name, {"custom" => {"_0" => "my-label"}}. Code reading hash[:custom][:_0] should read hash[:custom]. The previous shape matched no other binding and no core output.

  • Breaking (PHP binding): EntityCategory, PiiCategory and OutputFormat change from constants-only classes to classes with static factories, because the old shape could not carry a payload at all: a caller-supplied label was silently discarded in both directions, so Custom($label) always round-tripped as an empty string. Use EntityCategory::custom($label) and EntityCategory::person() in place of the old Xberg\EntityCategory::PERSON constants; the label is readable from the readonly $custom property.

  • Breaking (Node binding): the JSON surface is camelCase throughout, nested types included, and FormatMetadata is a flat discriminated union keyed by formatType whose variant payload fields sit directly on the object ({ formatType: "excel", sheetCount: 2 }). The binding previously exposed two parallel shapes for one Rust type -- an idiomatic camelCase interface beside a snake_case structural twin -- and only the latter was reachable from a result.

  • Breaking (Node, Swift bindings): FormatMetadata now serializes flat in every binding -- {"format_type": "pdf", "page_count": 12, ...} -- matching what the core Rust enum has always serialized (#[serde(tag = "format_type")]), what the OpenAPI discriminator describes, and what the REST API serves. Two bindings disagreed with that wire and have been corrected:

    • Node nested the payload one level down under a property named for the variant, so doc.metadata.format.pdf.pageCount becomes doc.metadata.format.page_count. Note the field names are snake_case, unlike the camelCase Node uses elsewhere: the variants carry mutually incompatible field types (headers is string[] for text and an object array for HTML), so no single Node class can describe them and the value is passed through as serde emits it. format is typed as a discriminated union in index.d.ts, so narrowing on format_type still gives a fully typed payload. This also resolves the Node and WebAssembly bindings disagreeing with each other -- WASM was already passing serde's shape through, so the two now emit an identical format object for the same document.

    • Swift's FormatMetadata was a typealias to an opaque bridge class carrying no payload, so the JSON in Metadata.format could not be decoded into anything useful. It is now a real Codable/Sendable enum with one case per format, making the payload reachable:

      if let json = doc.metadata?.format,
         case .excel(let meta) = try formatMetadataFromJson(json) {
          print(meta.sheetCount)
      }

      Metadata.format still hands back the serialized JSON String; what changed is that formatMetadataFromJson now yields a pattern-matchable enum carrying the payload instead of an opaque handle. The wire was already correct here -- the Swift type system was the part that was missing.

    Python, Go, Java, C#, Kotlin, PHP and Ruby are unaffected on the wire: they either already emitted the flat shape or expose native per-variant accessors over it (#1594).

  • Breaking (Rust source, Java): ServerConfig adds job_timeout_secs. Exhaustive Rust struct literals must set the field or use ..ServerConfig::default(), and the Java record's canonical constructor gains a sixth component, so new ServerConfig(host, port, corsOrigins, maxRequestBodyBytes, maxMultipartFieldBytes) no longer compiles -- use ServerConfig.builder(), which is unaffected. Every other binding is source-compatible: the field is last and defaulted in the Python dataclass (= 600), the Kotlin data class (= 600L) and C# ({ get; init; } = 600); a defaulted keyword in Ruby and PHP; an optional pointer with omitempty in Go; and an additive xberg_server_config_job_timeout_secs getter in the C FFI (gated on api-types). Deserializing callers are unaffected everywhere -- the field carries #[serde(default)].

  • Public binding-facing structs in this crate are deliberately not #[non_exhaustive]: alef generates impl From<Mirror> for xberg::T with a struct literal in roughly ten binding crates, and #[non_exhaustive] forbids that cross-crate (E0639) -- including the ..Default::default() spread. Default plus #[serde(default)] is the forward-compatibility mechanism instead, and a field addition is recorded here as a labelled source break rather than prevented by the type system. #[non_exhaustive] is reserved for types excluded from binding generation.

  • Retroactive note for 1.1.4: Metadata#format in the Ruby binding changed shape and no changelog entry recorded it at the time. The format-specific payload had been nested under a _0 key (format.fetch(:_0).fetch(:title)); since 1.1.4 the payload's fields sit directly alongside the format_type tag (format.fetch(:title)). Ruby callers written against the older shape raise KeyError on _0. The binding has emitted the flat shape since 1.1.4; the generated Ruby e2e specs were still asserting the nested one, which is why this went unnoticed for two releases. Only the Ruby binding is affected. Part of GH#1594, which also tracks the Swift binding still discarding the payload entirely -- that half is not yet fixed.

5 days ago
kreuzberg

Benchmark Results 2026-09-10 (04336bd)

Comparative benchmark results from workflow run 34441359865.

Commit: 04336bd369a06829e03ef04f57ed3ab6d442dbb3 Date: 2026-09-10

5 days ago
kreuzberg

v1.1.5

Fixed

  • The Java binding compiles again. A method returning Option<Vec<u8>>Registry.sampleBytes is the only one today — was generated declaring Optional<byte[]> while returning a bare byte[], which javac rejects. 1.1.4 therefore published no Java artifact at all, and the spring-ai integration was blocked waiting on it. Fixed upstream in alef 0.85.12; this release regenerates on it.
5 days ago
kreuzberg

v1.1.4

Fixed

  • PDF reading order no longer tears a subscript off the symbol it names. Spans were ordered by the top of their bounding box, but a subscript is drawn 35-40% smaller than its base, so its top sits several points lower even though its baseline is a fraction of a point away. An unrelated span from the next column could sort between a base run and its own subscript, and the symbol the subscript names no longer existed anywhere in the output. Ordering now quantises the baseline into row bands before comparing horizontally, which is what every other caller of that comparator already did (GH#1600).
  • PDF table detection no longer bridges two separate tables across the graphics-free gap between them. A cell was built from intersection points alone, so a section heading printed in that gap was absorbed into one of the tables as a single-cell row. A candidate cell now also requires a drawn vertical rule spanning its own Y-range on both sides. The span tolerance is load-bearing: at the tighter X-axis value, rows of a table whose rules are inset by a few points are dropped (GH#1601).
  • PDF two-column detection no longer loses the page's split to a hanging-number indent. When any span straddled a correctly detected gutter, the split was replaced outright by the midpoint of the widest whole-page whitespace corridor — on a hanging-number layout, the indent between the numbers and the text. The reorder then hoisted every clause number out of its clause. A relocation is now rejected when it would move the split more than a quarter of the page width, which leaves every legitimate corridor move in the corpus intact (GH#1603).
  • PDF paragraph grouping no longer splits a numbered heading that wraps onto a shorter second line. The wrap exemption compared the two lines' right edges, but a heading fills its column on its FIRST line and the continuation is whatever is left over, so the metric was anti-correlated with the answer. The pair is now also exempt when the continuation opens lowercase AND the heading line reaches within a tolerance of the width of what would be merged onto it — the "fills its column" half the original rule stated but never measured. The lowercase test alone is not sufficient: body prose beginning lowercase under a complete numbered heading has the same signature (GH#1605).
  • PDF paragraph grouping now recognises a numbered heading whose line arrives as more than one text span. The break terms tested the predicate against a single span, so a heading set with a hanging section number — 3.1.7 in one span, its title in the next, on one baseline — never looked like a numbered heading and was left to the ordinary paragraph-gap rule. That rule needs a gap wider than ordinary line pitch, so every such heading whose body starts on the next line was welded into it. The line's spans are now re-joined before the predicate runs, which is what the continuation-merge pass already did (GH#1609).
  • PDF paragraph grouping now recognises a heading whose number is not its first token — ARTIKEL 1., Chapter 1, Appendix 1, Annex III, Exhibit A. The numbered-heading predicate is the only boundary signal available when a heading shares font, size, weight and leading with its neighbour, so a heading it could not see was welded onto the line above it, and a run of such headings collapsed into a single element. Recognition is by shape, not by a keyword list: one capitalised word standing in front of an enumerator. Prose that opens the same way — Artikel 12 van de wet is van toepassing. — stays prose, because behind a keyword the text after the enumerator must still be capitalised (GH#1608).
  • PDF heading detection no longer skips a numbered heading that is only two words long. Promotion of a bold, body-size line to a heading required more than two words — a floor that keeps short bold fragments out — and a numbered section title such as 3. PRIJZEN or 1. INTRODUCTION falls below it. Those lines stayed plain bold paragraphs, and a run of them was then coalesced into a single bold line in the rendered output, while the element stream still reported them separately. A numbered section heading is now exempt from the word-count floor; everything else still has to clear it (GH#1611).
  • OCR no longer adopts a markdown table rebuild that loses content. The rebuilt page replaced the original whenever it was merely non-empty, so a rebuild that dropped text still won. The rebuild is now rejected, with a warning naming both word counts, when it retains fewer words than the content it would replace (GH#1599).
  • PaddleOCR's default model_tier of mobile now resolves to the pp-ocrv6 small detection model (9.9 MB) rather than medium (62 MB). A tier named mobile silently loading the largest available model made a 21-page document take over ten minutes. small and medium share the same 18,708-character dictionary, so recognition coverage is unchanged. The documented model sizes were also wrong and have been corrected (GH#1602).
  • The PHP extension now loads on Debian 12 and other distributions built against GCC 12. The Linux publish runners ship GCC 13+, and the extension picked up a GLIBCXX_3.4.31 symbol from their libstdc++ while Debian 12 provides at most GLIBCXX_3.4.30. libstdc++ is now linked statically; the highest glibc requirement was already below Debian 12's (GH#1606).
6 days ago
kreuzberg

v1.1.3

Release v1.1.3

6 days ago
kreuzberg

v1.1.3

Added

  • Table.cell_styles and GridCell.heading_level / GridCell.style_name expose the paragraph style a DOCX table cell carries. A heading styled Heading1..Heading6 inside a w:tc — the banner row forms, questionnaires and datasheets use as a section title, and what Word's navigation pane and a TOC field treat as the document outline — previously reached every consumer as anonymous cell text. Cell text is deliberately unchanged: prefixing it with # would put a markdown heading inside a table cell. The style travels beside the text instead, so a caller can decide whether a heading 2 in a banner row is a section title or a column label. cell_styles is sparse and omitted entirely for tables whose cells carry no style, so ordinary tables serialise exactly as before (GH#1587).

Fixed

  • PDF text repair no longer welds two complete words into one. repair_ligature_spaces removes the space in …f + + i|l|f… to undo a real artefact — some PDFs decompose a ligature glyph and leave a spurious gap, so first arrives as f irst — but the same character pattern is an ordinary word boundary whenever a word ends in f and the next begins with i, l or f. The only guard was a hard-coded list of 33 short English words tested against the left token, so everything outside it welded, English included: relief for became relieffor and itself infringes became itselfinfringes. The space is now kept when either fragment is independently attested as a standalone word elsewhere in the same document, reusing the witness mechanism dehyphenation already applies. A fragment appearing only as one half of a candidate pair does not witness itself (GH#1591).

  • DOCX page counting no longer collapses a table onto one page. Word writes <w:lastRenderedPageBreak/> into every cell of a row that straddles a page boundary — one physical break, one marker per cell — and the duplicated markers were reduced to a single break, losing the originals with the duplicates. A seven-page document reported two. Breaks are now identified by table depth, row and cell, so a marker echoed across the cells of one row counts once while several breaks inside a single deep cell each still count (GH#1592).

  • PDF outline (bookmark) named destinations now resolve when the /Names -> /Dests name-tree key is UTF-16BE-with-BOM, the form Adobe Distiller writes. The lookup previously decoded the /Dest byte string with a lossy UTF-8 conversion before searching the tree; a name-tree key is a byte string compared by byte (ISO 32000-1 §7.9.6), not text, so the BOM was mangled into replacement characters and every such destination silently resolved to None, leaving the bookmark's dest as an unresolved Destination::Named with no page (GH#1589).

  • MimeDetectionPolicy::ContentOnly no longer rejects a legacy OLE2 Office document (.doc/.xls/.ppt) passed by path when the same bytes are accepted through the bytes API. Path-based content detection only sniffs the first 4 KB of a file, but an MS-CFB compound document cannot be typed from a prefix — identifying it means following the FAT sector chain to the root directory entry, which a truncated buffer cannot do. Detection now falls back to a structure-aware read of the file for a compound-file header that a 4 KB prefix left inconclusive, the same escape hatch a ZIP-based Office document already had for the same class of failure (GH#1590).

  • PDF table detection no longer invents a column boundary from a rule that stops short of the row band. BAND_RULE_SPAN_TOL was defined as SNAP_TOL, conflating two different questions: SNAP_TOL decides whether two coordinates are the same coordinate, while this one decides whether a vertical rule runs through a band. At 3pt an edge could fall short at each end and still count as spanning, so a band up to 6pt shorter than the rule beside it was cut where the drawn rule gave it no boundary. Those phantom columns are what let a band of prose inside a drawn frame split into cells and qualify as a table, which on the reported document cost page text. Now 1.0 and deliberately independent of SNAP_TOL (GH#1588).

  • Tesseract psm = 0 is now rejected at configuration validation. PSM 0 is Tesseract's PSM_OSD_ONLY — orientation and script detection with no character recognition — so it cannot satisfy a text-extraction request, and Tesseract emits no hOCR for it at all. Setting it previously succeeded while returning either a zero-length document or degraded, partially dropped text, depending on the Tesseract build, in both cases with no warning and at several times the cost of a normal run. The error now names the mode and points at 3 (auto), 6 (single block), and 11 (sparse text). Valid values are 1-13; omitting psm continues to let the pipeline choose (GH#1586).

7 days ago
kreuzberg

v1.1.2

This release contains a breaking public API change. TesseractConfig.psm is now optional. Callers that read or set it as a plain integer must handle None / null — see below.

Changed

  • Breaking: TesseractConfig.psm is now Option<i32> (null/None/absent in the bindings) and defaults to unset rather than to 3. This fixes supplying a TesseractConfig at all acting as a hidden behaviour switch: because several code paths keyed on the struct being absent, a caller who set one unrelated field — table detection, a preprocessing knob — silently lost the whole-image PSM 11, the vertical-language PSM 5, the layout-region PSM 6, and the sparse-text retry, and got Tesseract's PSM 3 instead. TesseractConfig() with default fields is now a no-op: the pipeline applies exactly the same automatic PSM it would with no TesseractConfig. An explicitly set psm is still honoured. Bindings that model psm as a plain integer expose a companion presence check (for example xberg_tesseract_config_has_psm in the C API), since a bare integer cannot distinguish "unset" from a real 0.

Fixed

  • Fixed a numbered or bulleted list on a scanned page being reconstructed as a table, replacing the list text with a mangled grid. A candidate region whose first column is list markers (1., a), ) end to end — the header cell included — is now rejected on the OCR routes. A genuine numbered table is unaffected: its first column carries a header label (Line, Item) above the numbers, which is what separates the two.
  • Fixed a table detected on a scanned page having its text returned twice — once as paragraphs, once as table cells — in the document content and element tree. This affected every output_format; "plain" only appeared to avoid it.
  • Fixed PdfConfig.top_margin_fraction / bottom_margin_fraction defaulting to 0.06/0.05 (6%/5%) since 1.1.0, which silently dropped OCR text — page titles included — in the top and bottom bands of every default-config scanned PDF page with no warning. Both now default to 0.0 (disabled); set them explicitly to filter header/footer content. The nonzero defaults also forced every default-config OCR page onto the lossy per-page route instead of a document-capable backend's whole-document path; that routing is restored too.
  • Fixed rendered PDF pages losing the Tesseract backend's own ProcessingWarnings (including the dictionary-filter removal notice) and OCR metadata (psm, language, tesseract_dict_invalid_word_ratio), both of which reached the caller for a standalone image but were silently dropped for the same page rendered from a PDF.
  • Fixed OcrConfig.language being discarded whenever a TesseractConfig was supplied, so a German document was OCR'd in English. One precedence rule now governs both Tesseract backends and the vertical-language check.
  • Fixed supplying any ImageExtractionConfig suppressing document-level OCR on scanned PDFs, which returned empty pages with only a debug log.
  • Fixed rendered PDF pages ignoring the configured render DPI. target_dpi, min_dpi, max_dpi, and auto_adjust_dpi are now honoured. With no configuration the default stays at 150 DPI, unchanged.
  • Fixed suspended hyphens being welded during text assembly, turning onderhouds- en into onderhoudsen. A hyphen is now joined only across a genuine visual line break, matching the rule the pipeline layer already applied.
  • Fixed an unruled full-width band in a ruled table being cut at column positions no rule gives it, splitting headings mid-word. A column boundary now counts only where a vertical edge actually spans the band.
  • Fixed every non-header table cell having its em-dashes, en-dashes and minus signs rewritten to an ASCII hyphen, the spaces around a hyphen collapsed, E-/E+ lowercased to e-/e+, and any cell consisting solely of a dash emptied. That normalisation is correct for a numeric column (an em-dash means nil, 1.5E-05 is an exponent, - 3 is -3) but corrupted prose tables, turning Functionaliteit—12 into Functionaliteit-12 and a part code HRE - HReco into HRe-HReco. It is now applied only to columns whose data cells are predominantly numeric (#1582).
  • Fixed the Windows PHP extension archives failing to publish at all. vendor-windows-native-closure.ps1 repacks a .zip with Compress-Archive, which runs no native command and so never sets $LASTEXITCODE; the release workflow gated on it, and an unset $LASTEXITCODE compares as non-zero, so every Windows archive was rejected immediately after being vendored successfully. Combined with an all-or-nothing matrix gate that withheld the release's PHP assets whenever any single leg failed, this left v1.1.0 and v1.1.1 with no PHP binaries at all. Both are fixed: the script now sets its exit contract explicitly, matching its sibling scripts, and the upload job now ships the archives from the legs that succeeded (#1585).
8 days ago
kreuzberg

v1.1.1

This release contains a breaking public API change. OutputFormat::Structured is renamed to OutputFormat::DocTags. Update any config, CLI invocation, or binding call using output_format = "structured" to "doctags".

Changed

  • Breaking: renamed OutputFormat::Structured to OutputFormat::DocTags across every binding and the output_format config field. The rename shipped in 1.1.0 but was only alluded to there, with no entry describing it; the variant was renamed, not removed, and is available as "doctags". An output_format of "structured" is not rejected — it resolves to a custom renderer of that name, which is not registered.
  • Breaking: the TypeScript and WebAssembly OutputFormat type is a string union again ("plain" | "markdown" | "djot" | "html" | "json" | "doctags" | ...), matching the serde wire format shared with the CLI, REST, MCP, config-file, and Go surfaces. 1.1.0 briefly published an object union ({ type: "markdown" }) for these two bindings only.

Fixed

  • Fixed PHP extension packaging, which produced no PIE archives for 1.1.0.
  • Fixed HEIC and AVIF decoding on the Linux (glibc) Node binding, which shipped a libheif built with no HEVC or AV1 decoder at all — every .heic and .avif input failed to decode while the heic feature still reported as present. The Elixir linux-gnu NIF carries the same working codec closure.
  • Fixed Elixir NIF publishing for linux-gnu and Windows, which produced no artifacts for 1.1.0 and left the Hex package at 1.0.14. The linux-gnu NIF is now built against the glibc 2.28 floor it claims to support; the artifacts published for 1.0.14 bundled HEIF codec libraries that required a newer glibc.
  • Fixed the Windows Hex package, which declared the x86_64-pc-windows-gnu target while CI built and published x86_64-pc-windows-msvc. RustlerPrecompiled resolves the msvc triple on Windows and rejects any triple the package does not declare, so mix deps.get failed with "precompiled NIF is not available for this target" even though the artifact existed. Windows users had to compile the NIF from source.

Security

  • The Linux binding images now verify a pinned SHA-256 for every vendored native dependency (libde265, libheif, ONNX Runtime) before building it, instead of trusting the download. These libraries are linked into the published Node and Elixir artifacts.