96 lines
5.2 KiB
Plaintext
96 lines
5.2 KiB
Plaintext
|
### Supported File Formats (90+)
|
||
|
|
|
||
|
|
90+ file formats across 8 major categories with intelligent format detection and comprehensive metadata extraction.
|
||
|
|
|
||
|
|
#### Office Documents
|
||
|
|
|
||
|
|
| Category | Formats | Capabilities |
|
||
|
|
|----------|---------|--------------|
|
||
|
|
| **Word Processing** | `.docx`, `.docm`, `.dotx`, `.dotm`, `.dot`, `.odt` | Full text, tables, images, metadata, styles |
|
||
|
|
| **Spreadsheets** | `.xlsx`, `.xlsm`, `.xlsb`, `.xls`, `.xla`, `.xlam`, `.xltm`, `.xltx`, `.xlt`, `.ods` | Sheet data, formulas, cell metadata, charts |
|
||
|
|
| **Presentations** | `.pptx`, `.pptm`, `.ppsx`, `.potx`, `.potm`, `.pot`, `.ppt` | Slides, speaker notes, images, metadata |
|
||
|
|
| **PDF** | `.pdf` | Text, tables, images, metadata, OCR support |
|
||
|
|
| **eBooks** | `.epub`, `.fb2` | Chapters, metadata, embedded resources |
|
||
|
|
| **Database** | `.dbf` | Table data extraction, field type support |
|
||
|
|
| **Hangul** | `.hwp`, `.hwpx` | Korean document format, text extraction |
|
||
|
|
|
||
|
|
#### Images (OCR-Enabled)
|
||
|
|
|
||
|
|
| Category | Formats | Features |
|
||
|
|
|----------|---------|----------|
|
||
|
|
| **Raster** | `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.bmp`, `.tiff`, `.tif` | OCR, table detection, EXIF metadata, dimensions, color space |
|
||
|
|
| **Advanced** | `.jp2`, `.jpx`, `.jpm`, `.mj2`, `.jbig2`, `.jb2`, `.pnm`, `.pbm`, `.pgm`, `.ppm` | OCR via hayro-jpeg2000 (pure Rust decoder), JBIG2 support, table detection, format-specific metadata |
|
||
|
|
| **Vector** | `.svg` | DOM parsing, embedded text, graphics metadata |
|
||
|
|
|
||
|
|
#### Web & Data
|
||
|
|
|
||
|
|
| Category | Formats | Features |
|
||
|
|
|----------|---------|----------|
|
||
|
|
| **Markup** | `.html`, `.htm`, `.xhtml`, `.xml`, `.svg` | DOM parsing, metadata (Open Graph, Twitter Card), link extraction |
|
||
|
|
| **Structured Data** | `.json`, `.yaml`, `.yml`, `.toml`, `.csv`, `.tsv` | Schema detection, nested structures, validation |
|
||
|
|
| **Text & Markdown** | `.txt`, `.md`, `.markdown`, `.djot`, `.rst`, `.org`, `.rtf` | CommonMark, GFM, Djot, reStructuredText, Org Mode |
|
||
|
|
|
||
|
|
#### Email & Archives
|
||
|
|
|
||
|
|
| Category | Formats | Features |
|
||
|
|
|----------|---------|----------|
|
||
|
|
| **Email** | `.eml`, `.msg` | Headers, body (HTML/plain), attachments, threading |
|
||
|
|
| **Archives** | `.zip`, `.tar`, `.tgz`, `.gz`, `.7z` | File listing, nested archives, metadata |
|
||
|
|
|
||
|
|
#### Academic & Scientific
|
||
|
|
|
||
|
|
| Category | Formats | Features |
|
||
|
|
|----------|---------|----------|
|
||
|
|
| **Citations** | `.bib`, `.biblatex`, `.ris`, `.nbib`, `.enw`, `.csl` | Structured parsing: RIS (structured), PubMed/MEDLINE, EndNote XML (structured), BibTeX, CSL JSON |
|
||
|
|
| **Scientific** | `.tex`, `.latex`, `.typst`, `.jats`, `.ipynb`, `.docbook` | LaTeX, Jupyter notebooks, PubMed JATS |
|
||
|
|
| **Documentation** | `.opml`, `.pod`, `.mdoc`, `.troff` | Technical documentation formats |
|
||
|
|
|
||
|
|
#### Code Intelligence (300+ Languages)
|
||
|
|
|
||
|
|
| Feature | Description |
|
||
|
|
|---------|-------------|
|
||
|
|
| **Structure Extraction** | Functions, classes, methods, structs, interfaces, enums |
|
||
|
|
| **Import/Export Analysis** | Module dependencies, re-exports, wildcard imports |
|
||
|
|
| **Symbol Extraction** | Variables, constants, type aliases, properties |
|
||
|
|
| **Docstring Parsing** | Google, NumPy, Sphinx, JSDoc, RustDoc, and 10+ formats |
|
||
|
|
| **Diagnostics** | Parse errors with line/column positions |
|
||
|
|
| **Syntax-Aware Chunking** | Split code by semantic boundaries, not arbitrary byte offsets |
|
||
|
|
|
||
|
|
Powered by [tree-sitter-language-pack](https://github.com/kreuzberg-dev/tree-sitter-language-pack) — [documentation](https://docs.tree-sitter-language-pack.kreuzberg.dev).
|
||
|
|
|
||
|
|
**[Complete Format Reference](https://docs.kreuzberg.dev/reference/formats/)**
|
||
|
|
|
||
|
|
### Key Capabilities
|
||
|
|
|
||
|
|
- **Text Extraction** - Extract all text content with position and formatting information
|
||
|
|
- **Metadata Extraction** - Retrieve document properties, creation date, author, etc.
|
||
|
|
- **Table Extraction** - Parse tables with structure and cell content preservation
|
||
|
|
- **Image Extraction** - Extract embedded images and render page previews
|
||
|
|
- **OCR Support** - Integrate multiple OCR backends for scanned documents
|
||
|
|
{% if features.async %}
|
||
|
|
- **Async/Await** - Non-blocking document processing with concurrent operations
|
||
|
|
{% endif %}
|
||
|
|
{% if features.plugin_system %}
|
||
|
|
- **Plugin System** - Extensible post-processing for custom text transformation
|
||
|
|
{% endif %}
|
||
|
|
{% if features.embeddings %}
|
||
|
|
- **Embeddings** - Generate vector embeddings using ONNX Runtime models
|
||
|
|
{% endif %}
|
||
|
|
- **Batch Processing** - Efficiently process multiple documents in parallel
|
||
|
|
- **Memory Efficient** - Stream large files without loading entirely into memory
|
||
|
|
- **Language Detection** - Detect and support multiple languages in documents
|
||
|
|
{% if features.code_intelligence %}
|
||
|
|
- **Code Intelligence** - Extract structure, imports, exports, symbols, and docstrings from [300+ programming languages](https://docs.tree-sitter-language-pack.kreuzberg.dev) via tree-sitter
|
||
|
|
{% endif %}
|
||
|
|
- **Configuration** - Fine-grained control over extraction behavior
|
||
|
|
|
||
|
|
### Performance Characteristics
|
||
|
|
|
||
|
|
| Format | Speed | Memory | Notes |
|
||
|
|
|--------|-------|--------|-------|
|
||
|
|
| **PDF (text)** | 10-100 MB/s | ~50MB per doc | Fastest extraction |
|
||
|
|
| **Office docs** | 20-200 MB/s | ~100MB per doc | DOCX, XLSX, PPTX |
|
||
|
|
| **Images (OCR)** | 1-5 MB/s | Variable | Depends on OCR backend |
|
||
|
|
| **Archives** | 5-50 MB/s | ~200MB per doc | ZIP, TAR, etc. |
|
||
|
|
| **Web formats** | 50-200 MB/s | Streaming | HTML, XML, JSON |
|