Data platforms & pipelines
Getting data in, checking it, and keeping it correct — from a handful of spreadsheets to millions of rows a day.
Data processing · Data standards & validation · Transactional databases · Web data acquisition
Data & storage
Data processing
Columnar dataframes instead of ad-hoc scripts, so a million rows costs what a thousand does.
- Polars
- Parquet
- Arrow
- Data Package
What it is
Reading, reshaping and validating datasets that are too big for a spreadsheet and too irregular for a single SQL statement — the work between a raw file arriving and a table something can be built on.
Why we chose it
Polars gives us columnar, multi-threaded dataframes with a lazy query planner, in both TypeScript and Python, so the same transformation logic is available to a pipeline and to an API. Parquet and Arrow are the formats we move data in, because they keep types intact across process boundaries where CSV silently loses them. The Data Package standard wraps the result, so a dataset arrives with its schema attached rather than described in a README beside it.
What we build with it
Where we use it
Behind every ingestion pipeline we run, and inside the open-source data tooling we publish — the same engine serves both, which is why the tooling stays realistic.
captureapplication.fairspec.org
Fairspec — data framework in TypeScript and Python
A specification for describing tabular datasets, with reference implementations in TypeScript and Python.
we author
capturemycarro.app
Mycarro — car marketplace and dealer CRM
A used-car marketplace live in Portugal. Consumer search over listings gathered from across the market, a dealer back office for leads, inventory and invoicing, and a mobile app — built end to end.
we built
Get in touch
Tell us what you need built.
We are a small team in Portugal, working London hours. A site, an app, an internal tool, or the platform behind them — start with a sentence about the problem and we will take it from there.