fastrecon: Python Data Reconciliation Library for SQL, CSV, Parquet & ETL Testing

fastrecon is a free, open-source Python data reconciliation library for comparing two tabular datasets. It supports SQL tables, SQL queries, CSV, DAT (pipe-delimited), Parquet, JSON/NDJSON, Excel (.xlsx), XML, Avro, ORC, fixed-width and mainframe files through a unified DuckDB + Polars + PyArrow engine. Any combination of those 11 source types works with a single fr.compare() call.

Run pip install fastrecon. For database-specific connectors use extras: pip install "fastrecon[postgres]" for PostgreSQL, "fastrecon[snowflake]" for Snowflake, "fastrecon[databricks]" for Databricks, or "fastrecon[all-files]" for Excel, Avro, ORC and XML support.

Use fr.compare(left=fr.CsvFile("source.csv"), right=fr.SqlTable(conn="postgresql://...", table="my_table"), keys=["id"]). fastrecon streams the SQL table through DuckDB and diffs it row by row against the CSV. The same API works for all 31 source-type combinations. See example 10 →

Yes. fr.compare(left=fr.ParquetFile("data.parquet"), right=fr.SqlTable(conn=..., table="my_table"), keys=["id"]) diffs the Parquet file against the SQL table. The same API covers Parquet vs SQL Query too. See example 16 →

fastrecon supports 11 source types: CSV, DAT (pipe-delimited via CsvFile with options={"delim":"|"}), Fixed Width, Parquet, JSON/NDJSON, Excel (.xlsx), XML, Avro, ORC, SQL Table (any SQLAlchemy URI), and SQL Query. The reconciliation matrix shows all 31 distinct pair combinations.

fastrecon has 7 compare modes: keyed (full row-level diff, default), hash (single xxhash64 fingerprint per side), schema (column names and types only), rowcount (schema + counts), names_only (cheapest shape check), sampled (keyed diff on a random sample), and profile (per-column min/max/distinct/null statistics). See the modes table →

Yes. Pass exclude_columns=["updated_at", "load_ts"] to drop audit columns, and tolerances={"amount": 0.01} to allow a ±0.01 absolute difference on numeric columns. Both can be passed directly to compare() or bundled in a ReconConfig object for reuse. See example 32 →

ReconResult exposes six renderers: to_text() for a human-readable terminal report, to_json() for structured output, to_html() for browser reports, to_junit() for JUnit XML (Jenkins, GitHub Actions, etc.), to_csv() for spreadsheet export, and to_dict() for programmatic access. See example 35 →

Yes. Any database with a SQLAlchemy driver works with SqlTable and SqlQuery. Install the matching extra: pip install "fastrecon[snowflake]" for Snowflake or "fastrecon[databricks]" for Databricks. PostgreSQL, MySQL, SQL Server and SQLite work out of the box. ODBC and JDBC sources are also supported via OdbcQuery and JdbcQuery.

Yes. fastrecon is designed for automated ETL testing and data pipeline validation. result.exit_code returns 0 for MATCH, 1 for MISMATCH and 2 for ERROR — making it trivial to fail a CI step. The to_junit() renderer produces JUnit XML that integrates with Jenkins, GitHub Actions, GitLab CI and other CI/CD systems. Failures are always returned as a ReconResult rather than raised as exceptions, so one bad comparison never aborts a batch run.

Use CsvFile with a delim option: fr.compare(left=fr.CsvFile("source.dat", options={"delim":"|"}), right=fr.CsvFile("target.dat", options={"delim":"|"}), keys=["id"]). The same approach works for any delimiter. See example 12 →

fastrecon requires Python 3.9 or later. It is tested on Python 3.9, 3.10, 3.11, and 3.12 on Linux, macOS and Windows.