Skip to content

Nyctea

Nyctea

Polars-native data validation with a composable validator pipeline.

Get started View on GitHub 

uv add nyctea

pip install nyctea

Features

  • Composable pipeline

    Add, remove, or reorder validation phases. Inject custom logic anywhere without forking the library.

  • Validator registry

    Register parsers and checks by name. Share a Registry across many schemas.

  • Lazy by default

    All phases run on LazyFrame. Designed for datasets larger than RAM.

  • Schema as code or YAML

    Define schemas in Python dicts, YAML, or JSON. Load from file or build programmatically.

  • Parse then validate

    Transform columns before checking. The output frame is already clean and typed.

  • Structured errors

    Three reporting modes: summary counts, row indices, or cell values with ErrorReportConfig.

Quick example

from nyctea import SchemaModel, Registry, register_builtins
import polars as pl

schema = SchemaModel.from_dict({
    "columns": {
        "age":  {
            "dtype": "Int64",
            "nullable": False,
            "checks": [{"name": "min_value", "args": {"min": 0}}],
        },
        "name": {
            "dtype": "Utf8",
            "nullable": False,
            "parsers": [{"name": "strip"}, {"name": "lower"}],
        },
    }
})

registry = Registry()
register_builtins(registry)

df = pl.read_csv("data.csv")
result = schema.validate(df, registry)

print(result.report.summary())  # (1)!
result.errors                   # (2)!
  1. Human-readable summary: rows processed, rows valid, per-column stats.
  2. Structured DataFrame with column, check, and failure count per row.

Browse the docs

  • User Guides

    Get up and running. Learn schemas, parsers, checks, and the registry.

    Guides

  • API Reference

    Full reference for SchemaModel, Registry, ValidationResult, and more.

    API

  • Development

    Architecture decisions, contributing, and release notes.

    Development

Why Nyctea?

Verified against Pandera 0.29.0 · Patito 0.8.6 · Dataframely 2.7.0 · Great Expectations 1.15.0.

Nyctea Pandera 0.29 Patito 0.8.6 Dataframely 2.7
Polars-native Partial (multi-backend) (Polars-only)
LazyFrame accepted Partial¹ Partial (eager=False)
True out-of-core / streaming (deferred²)
Validator registry (Polars) (pandas only)
Composable pipeline phases
Parsers on Polars backend (pandas only)

Footnotes

¹ Pandera LazyFrame requires PANDERA_VALIDATION_DEPTH=SCHEMA_AND_DATA for full data validation, which forces collect(). "Lazy validation" in Pandera means deferred error collection, not lazy execution.

² Dataframely eager=False defers validation to collect() time. Data must still fit in memory.

Great Expectations 1.15 has no Polars support. The community request was closed as "not planned" in August 2024.

Full comparison and citations