kreuzberg-dev/kreuzberg
 Watch   
 Star   
 Fork   
16 days ago
kreuzberg

v1.3.10

Fixed

  • (pdf): a margin-note column beside a justified main column is read as its own block. A main column set as one span per line and wider than 55% of the page (375 pt of 595 pt) had every line discarded as full-width furniture, so the two-column repair never saw a column beside the notes, and its bands were cut at every body line. The margin label was placed after the first body line and the following lines alternated between the columns. Spans that do not cross the split are no longer furniture; the same holds with the notes on the left. (GH#2109)
  • (pdf): prose that runs across a table's inner gutter is no longer split between columns. A bordered two-column table beside a margin note let the table's gutter become a page-level column cut, and the full-width paragraphs above and below it were scattered over both groups, so the opening paragraph was read after a lower heading and a short last line such as 2017): came before its own paragraph. A column cut is now refused when two or more continuous lines have at least 60 pt of ink on each side of it. (GH#2110)
  • (deps): the unmaintained unsafe-libyaml is gone. YAML parsing moved from serde_yaml_ng to yaml_serde 0.10.7, clearing RUSTSEC-2024-0449 without an ignore entry. (GH#2108)
  • (deps): ort is pinned to exactly 2.0.0-rc.13. The caret requirement let a fresh resolve pick rc.14, whose session builder needs an Environment, breaking downstream builds.
  • (swift): the iOS libraries build again. A workflow-wide MACOSX_DEPLOYMENT_TARGET made the Tesseract CMake build compile iOS objects as macOS, so linking failed; xberg-tesseract now ignores the variable for iOS targets.
  • (elixir): the NIF load check accepts BEAM-provided enif_* symbols. Release builds of the NIF no longer fail the dlopen probe on symbols only the BEAM supplies.
  • (php): the Linux extension links ONNX Runtime dynamically. The statically linked runtime needed GCC 13 libstdc++ symbols that the zigbuild toolchain replaces with libc++, so xberg.so could not load on Debian 12. libonnxruntime now ships beside xberg.so, and the release gate fails on unversioned undefined C++ symbols.
  • (zig): trait-bridge callbacks no longer crash when they return a string. The generated vtable thunks declared char ** out-parameters as 16-byte optionals, so registering a post-processor segfaulted. Regenerated with alef 0.110.6.
  • (ci): lint and e2e jobs are green again. The ai-rulez generator is built with GOWORK=off, governance uses ai-rulez 5.1.0, the integration npm lockfiles match vitest 5, the Ruby e2e lockfile matches the gemspec, and the bounded-read and strict_parse tests follow zip 9.0.2 and the plain-text fallback.
16 days ago
kreuzberg

v1.3.10

Fixed

  • (deps): the unmaintained unsafe-libyaml is gone. YAML parsing moved from serde_yaml_ng to yaml_serde 0.10.7, clearing RUSTSEC-2024-0449 without an ignore entry. (GH#2108)
  • (deps): ort is pinned to exactly 2.0.0-rc.13. The caret requirement let a fresh resolve pick rc.14, whose session builder needs an Environment, breaking downstream builds.
  • (swift): the iOS libraries build again. A workflow-wide MACOSX_DEPLOYMENT_TARGET made the Tesseract CMake build compile iOS objects as macOS, so linking failed; xberg-tesseract now ignores the variable for iOS targets.
  • (elixir): the NIF load check accepts BEAM-provided enif_* symbols. Release builds of the NIF no longer fail the dlopen probe on symbols only the BEAM supplies.
  • (php): the Linux extension links ONNX Runtime dynamically. The statically linked runtime needed GCC 13 libstdc++ symbols that the zigbuild toolchain replaces with libc++, so xberg.so could not load on Debian 12. libonnxruntime now ships beside xberg.so, and the release gate fails on unversioned undefined C++ symbols.
  • (zig): trait-bridge callbacks no longer crash when they return a string. The generated vtable thunks declared char ** out-parameters as 16-byte optionals, so registering a post-processor segfaulted. Regenerated with alef 0.110.6.
  • (ci): lint and e2e jobs are green again. The ai-rulez generator is built with GOWORK=off, governance uses ai-rulez 5.1.0, the integration npm lockfiles match vitest 5, the Ruby e2e lockfile matches the gemspec, and the bounded-read and strict_parse tests follow zip 9.0.2 and the plain-text fallback.
17 days ago
kreuzberg

v1.3.9

Added

  • (config): strict_parse controls what happens when a text-like document cannot be parsed in its own format. By default (false), a YAML, JSON, TOML or XML file that fails to parse, an XML or SVG file that trips security_limits.max_entity_length, and a CSV or TSV file that trips security_limits.max_table_cells are extracted as plain text instead of failing, and the reason is recorded in processing_warnings (source parse-fallback). Plain-text recovery is truncated to security_limits.max_content_size with a warning. Set strict_parse = true (CLI --strict-parse true, environment XBERG_STRICT_PARSE) to keep the error. The limits are not loosened, and limits aimed at hostile input (archive size, nesting, XML depth) still fail the extraction.

Changed

  • Updated dependencies, including tree-sitter-language-pack 1.22.1 (25 refreshed grammars), zip 9.0.2, biblib 0.10, html-escape 0.3 and unhwp 0.19. ort stays at 2.0.0-rc.13 because rc.14 requires an Environment for every session builder.

Fixed

  • (extraction): a Helm template or other unparseable YAML, JSON, TOML or XML file no longer fails the extraction. The content is returned as plain text with a warning; strict_parse = true restores the error. (GH#2100)
  • (extraction): an oversized XML entity or table no longer discards a text file. A CSV over security_limits.max_table_cells or an XML or SVG file over security_limits.max_entity_length is extracted as plain text with a warning naming the limit that tripped; both limits remain adjustable through security_limits. (GH#2101)
  • (release): the Helm chart is published even when the CLI image build fails. The chart publish was gated on the aggregate result of the image manifest job, so a CLI image leg timing out skipped the chart for 1.3.6 to 1.3.8 while the container images shipped. It is now gated on the xberg and xberg:<version>-core manifests resolving, the only images the chart references. (GH#2106)
  • (pdf): image bounding boxes are correct for images drawn inside a Form XObject with a /Matrix. The form /Matrix was applied after the page transform instead of before it, so the form's page translation was scaled by its own matrix and ExtractedImage.bounding_box landed in the wrong place (a regression since 1.2.7). (GH#2103)
  • (pdf): a narrow margin-note column is read before the main text again. A column of notes set on its own leading beside the main text (no note line shares a baseline with a body line) gave the two-column repair too few gutter votes once a table's grid rows stopped voting, so the page was read line by line across both columns and the margin label was welded to the main heading. A page whose only whitespace corridor is a gutter narrower than the native XY-cut's 15 pt valley is now split at that corridor. (GH#2102)
  • (rendering): a block with many inline annotations renders in linear time. Building the inline nodes of one paragraph rescanned every later annotation for each annotation, so Markdown or chunked extraction of a block dense with emphasis, code spans and links grew quadratically (1 MiB: 29 s, 4 MiB: over 400 s). The scan now stops at the end of the annotated span. The output does not change. (GH#2099)
  • (pdf): two-column pages whose column baselines differ by up to 1pt are read in column order again. The XY-cut line grouping measured gaps between spans in y order, so a right-column line set slightly higher sorted first and merged with the left column's line into one line, which the column split then sent to one side. Spans are now ordered by x before gaps are measured, and a gap is measured from the line's rightmost edge so far. A column cut that would separate short label/value rows whose baselines differ slightly is refused, so key-value rows and table rows read row by row. (GH#2097)
  • (pptx, ocr): plain output of a PPTX file has the OCR text of a picture on a slide. The text is in content and in the content of the slide one time. Before, OCR read the picture and the text was in no part of the plain output. The text is also there when images.inject_placeholders is false in Markdown, Djot and HTML output. (GH#2086)
  • (ocr): text that has the form of a Markdown image reference does not repeat the OCR text of a picture. When a document had a paragraph such as ![label](label.png) as text, and a picture whose OCR text is shown without a placeholder, the text of the picture was in content two times. A PDF file with the default settings is such a document. The text is now there one time. (GH#2093)
  • (ppt, ocr): the content of a slide in pages has the OCR text of a picture in plain output. OCR read the picture on a PPT slide and the text was in content of plain output, but not in the content of the slide. Page content that is built from the elements of a page now has the text one time, after the text of the picture, when content is the plain rendering: in plain output, and in the output of a custom format name that has no renderer. Page content is built that way when the extractor stores no pages: a PPT file is such a document, and a PDF file is one when page extraction is not set. The page content of Markdown, Djot, HTML, JSON and a custom renderer does not change. (GH#2088)
  • (pdf, ocr): a failed OCR fallback for a whole PDF now reports its failed pages in ocr_page_failures. When the OCR run of a whole document failed and the native text was kept, the result had a warning and no record. Each page that has no native text now has a record, as it does when OCR runs on the scanned pages only. A page that has native text has no record. A page that failed in one OCR pipeline stage and got no text from a later stage had no record either, and now has one. (GH#2083)
19 days ago
kreuzberg

v1.3.8

Release v1.3.8

21 days ago
kreuzberg

v1.3.7

Added

  • (pdf): pre-extracted spans can use canonical page reading order. order_page_spans applies tagged structure, article-thread, and geometric ordering without parsing page content a second time, including for layer-filtered text.
  • (ocr): failed PDF OCR pages are reported as a typed list in every binding. ExtractedDocument has a new ocr_page_failures field. It is absent (None) when no page failed. Each OcrPageFailure gives the 1-based page, the backend error text, and recovered, which is true when the page still has content from native text or from its embedded images. When an automatic OCR run of selected pages fails as a whole, each selected page has a record. The records were a JSON array under metadata.additional["ocr_page_failures"]; that key is no longer set. The warnings do not change. (GH#2064)

Changed

  • (benchmarks): cold-process and warm in-process latency are reported as separate regimes. Opt-in steady-state Xberg runs retain model sessions across iterations, publish a separate cold-start probe, and remain excluded from cross-framework rankings until competitors expose equivalent persistent-process adapters.
  • (batch): default in-memory extraction avoids no-op cache work. Engines without an injected cache backend no longer hash every byte input or serialize completed results for a cache that discards them, reducing CPU time and transient allocations without changing extractor or OCR cache behavior.

Fixed

  • (node): extraction works with AsyncLocalStorage and async hooks enabled. Node bindings heap-pin extraction futures before creating their JavaScript promises, avoiding a synchronous stack overflow on Node 22 and later.

  • (pdf, ocr): Arabic and Hebrew extraction preserve logical reading order. Tagged native PDFs use trustworthy structure order without reversing table cells, and Tesseract output orders mixed-direction lines and tables from their local text direction and geometry. Sparse whole-image RTL OCR also retries a single-block segmentation mode when it recovers more strong-script tokens without materially reducing confidence, and mixed Arabic/Latin scans recover identifiers only when a bounded English crop provides compatible evidence.

  • (benchmarks): benchmark provenance is bound to the executable that actually ran. Clean-checkout runs now reject stale Xberg binaries whose embedded build identifier does not match the repository commit, and machine-readable xberg version output includes that identifier for auditability.

  • (doc): legacy Word documents preserve headers, footers, footnotes, comments, and text boxes. Non-body stories are now emitted with their document layers and respect the existing header, footer, and footnote filters instead of being dropped whenever the document also contains body text. (GH#2054)

  • (php): Linux packages run on Debian 12 and other glibc 2.36 systems. Release artifacts are built against the declared ABI floor, reject newer GLIBC, GLIBCXX, or CXXABI requirements, and are load-tested in Debian 12 before publication. (GH#2060)

  • (ocr): columned prose and punctuation fragments are no longer fabricated as tables. Long, sparse newsletter columns remain ordinary text in both layout and non-layout extraction, while punctuation-only scan regions fall back to paragraph output without losing recognized content.

  • (ocr): layout extraction preserves scanned-PDF recognition settings. Scanned PDFs no longer force Tesseract into whole-image single-block segmentation when layout assembly has detected regions, while known scans keep their sparse-text segmentation and source-resolution preprocessing hints.

  • (typst): tables wrapped in figures are preserved. A #table(...) nested inside a #figure(...), including through an align(...) wrapper, now reaches structured and rendered output instead of being discarded with the figure body.

  • (latex): multiline section titles remain headings. A section command whose braced title spans physical lines now produces one heading with its label anchor instead of flattened paragraph text and command fragments.

  • (python, pdf): Python post-processors preserve formatted output and hide internal PDF heading annotations. Returning an ExtractedDocument from a Python post-processor no longer replaces Rust-only document state, so a no-op processor keeps rendered headings and chunks. Font-size annotations used for native PDF heading detection are also excluded from plain text and keyword extraction. (GH#2050)

  • (pdf): conflicting JPEG 2000 palette color spaces fail consistently across architectures. Native PDF extraction now rejects a codestream palette whose component count conflicts with an explicit /ColorSpace, instead of relying on architecture-dependent decoder rounding that could accept the same malformed image on ARM and reject it on x86.

  • (zig): generated bindings compile with Zig 0.17. String duplication now uses the Zig 0.17 sentinel API, and plugin vtables cross the C ABI through layout-compatible pointers instead of rejected value bitcasts.

21 days ago
kreuzberg

Benchmark Results 2026-10-09 (bca0e2c)

Comparative benchmark results from workflow run 37864705744.

Commit: bca0e2c12ba0ad90c045e31db814876487b2b56b Date: 2026-10-09

22 days ago
kreuzberg

Benchmark Results 2026-10-08 (465445b)

Comparative benchmark results from workflow run 37719232667.

Commit: 465445b016c5b447b5cecd863864190a9e52c6e7 Date: 2026-10-08

23 days ago
kreuzberg

v1.3.6

Changed

  • (zig): the Zig binding now requires Zig 0.17.0. build.zig.zon declares .minimum_zig_version = "0.17.0" and the package is built and tested on Zig 0.17.
  • (pdf): lossy JPEG 2000 images now decode to the same pixels as OpenJPEG. Since 1.3.5 the JPEG 2000 decoder applies the standard's reconstruction offset for lossy coefficients (ISO 15444-1 E-6..E-8); earlier releases decoded such images slightly darker. Rendered pages and OCR input change accordingly. On scans with grey shaded table rows, OCR without normalize_shaded_rows can now recover fewer table rows than in 1.3.4; with normalize_shaded_rows enabled, results shift by only a few values.

Fixed

  • (docx): a footnote and an endnote with the same number are kept apart. Word numbers footnotes and endnotes independently, and both were keyed fn<id>, so in Markdown, HTML and Djot output the first endnote replaced the first footnote and both references pointed at it. Endnotes are now keyed en<id>, as in the ODT and WordPerfect extractors.
  • (pdf): mixed-rotation pages preserve word boundaries in upright text. Native extraction no longer rewrites an entire page through the rotated-run assembler when the upright content spans multiple rows, which previously joined adjacent words in reports with rotated footer text. (GH#2049)
24 days ago
kreuzberg

v1.3.5

What's Changed

New Contributors

Full Changelog: https://github.com/xberg-io/xberg/compare/v1.2.4...v1.3.5

25 days ago
kreuzberg

v1.3.4

What's Changed

New Contributors

Full Changelog: https://github.com/xberg-io/xberg/compare/v1.2.4...v1.3.4