Total Scripts: 41 (27 Bash + 14 PowerShell)
Documentation: See SCRIPT_MAPPING.md for detailed workflow-to-script mapping
All Scripts: Production-ready with proper error handling and documentation

Directory Structure

scripts/ci/
├── README.md               ← This file
├── SCRIPT_MAPPING.md       ← Detailed workflow-to-script mapping guide
├── docker/                 ← Docker image build and test scripts
├── go/                     ← Go bindings scripts
├── java/                   ← Java bindings scripts
├── node/                   ← Node/TypeScript NAPI scripts
├── python/                 ← Python wheel build scripts
├── ruby/                   ← Ruby gem build scripts
├── rust/                   ← Rust core and CLI scripts
├── csharp/                 ← C# bindings scripts
└── validate/               ← Validation and linting scripts

Quick Start

Running a Script

Bash scripts:

./scripts/ci/docker/build-image.sh core
./scripts/ci/python/run-tests.sh true

PowerShell scripts:

& ./scripts/ci/go/build-ffi.ps1
& ./scripts/ci/rust/package-cli-windows.ps1 -Target "x86_64-pc-windows-msvc"

Sourcing Scripts

For library path setup scripts:

source ./scripts/lib/library-paths.sh
setup_all_library_paths
./scripts/ci/python/run-tests.sh true

Scripts by Workflow

Docker (`docker/`)

free-disk-space.sh - Clean up CI disk space
build-image.sh - Build Docker image variant
check-image-size.sh - Validate image size constraints
save-image.sh - Save Docker image as tar.gz artifact
collect-logs.sh - Collect container logs on failure
cleanup.sh - Clean up Docker resources
summary.sh - Print test summary

Go (`go/`)

build-ffi.sh - Build FFI library (Unix)
build-ffi.ps1 - Build FFI library (Windows)
build-bindings.sh - Build Go bindings with CGO (Unix)
build-bindings.ps1 - Build Go bindings with CGO (Windows)
reorganize-libraries.ps1 - Reorganize FFI libraries for Windows
run-tests.sh - Run Go tests with library paths

Java (`java/`)

build-java.sh - Build Java bindings with Maven
run-tests.sh - Run Java tests with Maven

Node/TypeScript (`node/`)

build-napi.sh - Build NAPI bindings with artifact collection
unpack-bindings.sh - Unpack and install bindings from tarball

Python (`python/`)

clean-artifacts.sh - Clean previous wheel artifacts
smoke-test-wheel.sh - Test wheel installation
install-wheel.sh - Install platform-specific wheel
run-tests.sh - Run tests with optional coverage

Ruby (`ruby/`)

install-ruby-deps.sh - Install bundle dependencies (Unix)
install-ruby-deps.ps1 - Install bundle dependencies (Windows)
vendor-kreuzberg-core.py - Vendor core crate for packaging
configure-bindgen-windows.ps1 - Configure bindgen headers (Windows)
configure-tesseract-windows.ps1 - Configure Tesseract (Windows)
build-gem.sh - Build Ruby gem
install-gem.sh - Install built gem
compile-extension.sh - Compile native extension
run-tests.sh - Run RSpec tests

Rust (`rust/`)

configure-bindgen-windows.ps1 - Configure bindgen headers (Windows)
run-unit-tests.sh - Run Rust unit tests
package-cli-unix.sh - Package CLI as tar.gz (Unix)
package-cli-windows.ps1 - Package CLI as zip (Windows)
test-cli-unix.sh - Test CLI binary (Unix)
test-cli-windows.ps1 - Test CLI binary (Windows)

C# (`csharp/`)

build-csharp.sh - Build C# bindings with dotnet
run-tests.sh - Run C# tests with dotnet

Validate (`validate/`)

run-lint.sh - Run all linting and validation checks via Task

Features

Error Handling

All Bash scripts use set -euo pipefail
All PowerShell scripts use Set-StrictMode and error action preferences
Proper exit codes and error messages
Usage information for incorrect arguments

Documentation

Every script has a descriptive header
Purpose and usage clearly stated
Which CI workflow step uses it
Argument documentation

Platform Support

Windows-specific operations via PowerShell (.ps1)
Unix operations via Bash (.sh)
Cross-platform scripts detect OS and adjust behavior
Library path setup scripts handle Windows/Linux/macOS

Reusability

library-paths.sh (scripts/lib/) - Shared by all workflows for native library configuration
configure-bindgen-windows.ps1 used by Ruby and Rust
Common patterns consolidated into single scripts

Detailed Documentation

For comprehensive workflow-to-script mapping and usage examples, see SCRIPT_MAPPING.md.

Usage in Workflows

Example: ci-docker.yaml

Before (inline commands):

- name: Free up disk space
  run: |
    echo "=== Initial disk space ==="
    df -h /
    echo "=== Removing unnecessary packages ==="
    sudo rm -rf /usr/share/dotnet
    # ... 30+ lines of commands ...

After (using script):

- name: Free up disk space
  run: ./scripts/ci/docker/free-disk-space.sh

Example: ci-python.yaml

Before (inline commands):

- name: Run Python tests
  run: |
    cd packages/python
    if [ "${{ matrix.coverage }}" = "true" ]; then
      uv run pytest -vv --cov=kreuzberg --cov-report=lcov:coverage.lcov ...
    else
      uv run pytest -vv --reruns 1 --reruns-delay 1
    fi

After (using script):

- name: Run Python tests
  run: ./scripts/ci/python/run-tests.sh ${{ matrix.coverage }}

Testing Scripts Locally

You can test scripts locally before running in CI:

# Test Docker scripts
./scripts/ci/docker/free-disk-space.sh

# Test Python scripts
./scripts/ci/python/clean-artifacts.sh
./scripts/ci/python/run-tests.sh false

# Test Rust scripts
./scripts/ci/rust/run-unit-tests.sh

Shell Compatibility

Bash scripts: Compatible with bash 3.2+ (macOS) and bash 4.0+ (Linux)
PowerShell scripts: Compatible with PowerShell 5.1+ (Windows) and PowerShell Core 7+ (cross-platform)

Contributing

When adding new CI steps or modifying existing ones:

Extract the inline script into a separate file in the appropriate directory
Add proper error handling (set -euo pipefail for bash)
Include descriptive header comments
Update SCRIPT_MAPPING.md with the new mapping
Test the script locally before committing

Maintenance

Scripts should be reviewed and updated when:

Updating CI workflow logic
Changing build tools or versions
Improving error handling
Adding new platform support

See each script's header for detailed documentation on its purpose and usage.

README.md

CI Workflow Scripts

Overview