Moved. This repo has moved into the benzsevern/goldenmatch monorepo at packages/python/goldenflow (and packages/typescript/goldenflow)/. This repo is archived; new development happens in the monorepo.
GoldenFlow
Data transformation toolkit β standardize, reshape, and normalize messy data before it hits your pipeline.
Built by Ben Severn.
Works on files (CSV, Excel, Parquet), cloud storage (S3, GCS), or live databases. Zero-config mode auto-detects what needs fixing. One command to clean what GoldenCheck found and prep what GoldenMatch needs. Available in Python and TypeScript with full feature parity.

# Python
pip install goldenflow
goldenflow transform data.csv
# TypeScript / Node.js
npm install goldenflow
npx goldenflow-js transform data.csv
The Problem
Your data arrives broken in predictable ways:
- Phone numbers come in 15 different formats
- Dates are mixed between MM/DD/YYYY and YYYY-MM-DD
- Addresses have inconsistent abbreviations
- Column names don't match between systems ("fname" vs "first_name" vs "given_name")
- Values have leading whitespace, unicode garbage, smart quotes
- Categoricals are inconsistent ("USA", "US", "United States")
Every data engineer writes throwaway scripts to fix these. Every script is slightly different. None of them are reusable.
GoldenFlow makes the transforms reusable, composable, and automatic.
Quick Start
Python
pip install goldenflow
# Auto-transform (zero-config)
goldenflow transform messy_data.csv
# Try the demo first
goldenflow demo
goldenflow transform demo_data.csv -c demo_config.yaml
# With config
goldenflow learn messy_data.csv -o config.yaml
goldenflow transform messy_data.csv -c config.yaml
# Schema mapping
goldenflow map --source system_a.csv --target system_b.csv
# Full pipeline
goldencheck scan data.csv
goldenflow transform data.csv
goldenmatch dedupe data_transformed.csv
TypeScript / Node.js
npm install goldenflow
# Auto-transform (zero-config)
npx goldenflow-js transform messy_data.csv
# Try the demo first
npx goldenflow-js demo
npx goldenflow-js transform demo_data.csv -c demo_config.yaml
# With config
npx goldenflow-js learn messy_data.csv -o config.yaml
npx goldenflow-js transform messy_data.csv -c config.yaml
# Schema mapping
npx goldenflow-js map -s system_a.csv -t system_b.csv
Programmatic (TypeScript)
import { TransformEngine } from "goldenflow";
const result = new TransformEngine().transformDf([
{ name: " JOHN ", phone: "(555) 123-4567", email: "John@Example.COM" },
]);
console.log(result.rows[0]);
// { name: "JOHN", phone: "+15551234567", email: "john@example.com" }
Zero-Config Mode
goldenflow transform customers.csv
# or just:
goldenflow customers.csv
GoldenFlow profiles every column and applies safe transforms automatically:
- Strips whitespace and normalizes unicode
- Standardizes phone numbers to E.164 format
- Normalizes email casing
- Parses and standardizes date formats to ISO 8601
- Normalizes zip codes (zero-padding, strip +4)
- Replaces smart/curly quotes with straight quotes
- Auto-corrects categorical misspellings via fuzzy matching
Output: a clean CSV with a sidecar manifest showing every transform applied.
customers.csv -> customers_transformed.csv
-> customers_manifest.json
The manifest is an audit trail β what changed, why, and which rows were affected.
CLI Commands
GoldenFlow has 14 commands. The most common ones:
# Core transforms
goldenflow transform data.csv # Zero-config: auto-detect and fix
goldenflow transform data.csv -c config.yaml # Apply saved config
goldenflow transform data.csv --domain healthcare # Use a domain pack
goldenflow transform data.csv --strict # Fail on any transform error
goldenflow transform data.csv --llm # Enable LLM-enhanced corrections
goldenflow data.csv # Shorthand: auto-routes to transform
# Schema & profiling
goldenflow map -s a.csv -t b.csv # Auto-map schemas between files
goldenflow profile data.csv # Show column profiles
goldenflow learn data.csv -o config.yaml # Generate config from data patterns
goldenflow validate data.csv # Dry-run: show what would change
goldenflow diff before.csv after.csv # Compare pre/post transform
# Continuous & scheduled
goldenflow watch ./data/ # Auto-transform new/changed files
goldenflow schedule data.csv --every 1h # Run on a schedule (5m, 1h, 30s...)
goldenflow stream large_file.csv --chunk-size 50000 # Stream-process in batches
# Discovery & history
goldenflow init data.csv # Interactive setup wizard
goldenflow demo # Generate sample data to try
goldenflow history # Show recent transform runs
goldenflow history -n 50 # Last 50 runs
# Integrations
goldenflow interactive data.csv # Launch TUI
goldenflow serve # REST API for real-time transforms
goldenflow mcp-serve # MCP server for Claude Desktop
Default Routing
Running goldenflow <file> without a subcommand auto-routes to transform:
goldenflow customers.csv # equivalent to: goldenflow transform customers.csv
goldenflow - # read from stdin, write to stdout
Streaming
For files too large to load into memory, use StreamProcessor or the stream command:
goldenflow stream large_file.csv --chunk-size 50000
from goldenflow.streaming import StreamProcessor
processor = StreamProcessor(config=config)
# Process a single record
result = processor.transform_one({"name": " John ", "phone": "(555) 123-4567"})
# Process a batch
result = processor.transform_batch(df_batch)
# Stream a large file in chunks
for result in processor.stream_file("large_data.csv", chunk_size=10_000):
write_to_output(result.df)
print(f"Processed {processor.batches_processed} batches")
Cloud Connectors
GoldenFlow reads from and writes to S3 and Google Cloud Storage transparently:
# S3
goldenflow transform s3://my-bucket/raw/customers.csv -o s3://my-bucket/clean/
# GCS
goldenflow transform gs://my-bucket/data/records.csv
from goldenflow.connectors.s3 import read_s3, write_s3
from goldenflow.connectors.gcs import read_gcs, write_gcs
df = read_s3("s3://my-bucket/raw/customers.csv")
df = read_gcs("gs://my-bucket/data/records.csv")
Cloud paths are detected automatically β no extra flags needed.
Watch Mode
Auto-transform files as they arrive in a directory:
goldenflow watch ./data/
goldenflow watch ./incoming/ -c config.yaml -o ./processed/
GoldenFlow polls the directory and applies transforms to any new or changed files.
Scheduling
Run transforms on a repeating schedule:
goldenflow schedule data.csv --every 1h
goldenflow schedule data.csv --every 30m -c config.yaml -o ./output/
Supported intervals: 30s, 5m, 1h, 2h, etc.
Setup Wizard
Generate a YAML config interactively:
The wizard profiles your data, suggests transforms, and saves a goldenflow.yaml ready to use.
History
GoldenFlow tracks every transform run in ~/.goldenflow/history/:
goldenflow history # Last 20 runs
goldenflow history -n 50 # Last 50 runs
Each run record captures: source file, row count, transforms applied, errors, and duration.
Schema Mapping
When you need to merge data from different systems:
goldenflow map --source crm_export.csv --target warehouse_schema.csv
GoldenFlow auto-maps columns between schemas using name similarity and data profiling:
crm_export.csv warehouse schema
----- -----
email_address -> email (rename)
phone_number -> phone (rename)
fname -> first_name (alias match)
st -> state (alias match)
Ambiguous mappings get flagged for human review. Confident mappings apply automatically.
Domain Packs
Pre-configured transform sets for common industries. All 5 are now implemented: