Test-Driven Data Analysis

I gave a tutorial on TDDA at PyData London 2026 earlier this month.

As announced a few days ago, my book, Test-Driven Data Analysis, is now available for sale from all good booksellers and all sellers of good books, around the world.

Raw markdown for direct use in LLM context windows.

Raw markdown for direct use in LLM context windows.

Raw markdown for direct use in LLM context windows.

This blog has been quite quiet, but there is a great deal of news and it may be less quiet for a while.

Summary Computational documents attempt to guarantee that results included within them—such as graphs—correspond to the code and data claimed to generate them.

Almost all data scientists and data engineers have to work with flat files (CSV files) from time to time.

It is December 2024 as I write, and large language models (LLMs) are having an extended moment as I have been writing a book on tet-driven data analysis.

In a previous post, I discussed some of the dangers of challenges, dangers and weaknesses of Jupyter Notebooks, JupyterLabs and their ilk.

In his novel Stranger in a Strange Land, Robert Heinlein1 introduced the word grok.

I have long considered writing a post about the various problems I see with computational notebooks such as Jupyter Notebooks.

This is not, primarily, an article about TDDA, but I thought it was worth publishing here anyway.

PyData London had its tenth conference in 2024, and it was excellent.

I was at the tenth PyData London Conference last weekend, which was excellent, as always.

This is just a bit of fun, but I've always been interested in the different kinds of names allowed, encouraged, and used in different areas of computing and data.

TOMLParams is a new open-source library that helps Python developers to externalize parameters in TOML files.

I had the pleasure of discussing TDDA with Peter Schmidt on his Coding for Thought podcast.

On Episode 258 of Marco Arment and “Underscore” David Smith’s podcast Under the Radar, and then on Episode 516 of Marco & co’s Accidental Tech Podcast, Marco describes the fact that his data suggests that about 12% of his users don’t have logged-in iCloud…

We released version 2.0 of the Python TDDA library this week.

PDF Version (A4) PDF Version (Letter)

or Why Tests Spontanously Fail You might think that if you write a program, and don't change anything, then come back a day later (or a decade later) and run it with the same inputs, it would produce the same output.

This week, a client I'm working for received a large volume of data, and as usual the data was sent as "flat" files—or CSV (comma-separated values1) files, as they are more often called.

I've been dabbling in Swift—Apple's new-ish programming language—recently.

This exercise (video 2m 58s) shows a powerful way to run only a single test, or some subset of tests, by using the @tag decorator available in the TDDA library.

This exercise (video 3m 34s) shows a powerful way to run only a single test, or some subset of tests, by using the @tag decorator available in the TDDA library.

This exercise (video 8m 54s) shows how to migrate a test from using pytest directly to the exploiting the referencetest capabilities in the TDDA library.

This exercise (video 8m 53s) shows how to migrate a test from using unittest directly to the exploiting the referencetest capabilities in the TDDA library.

We've started producing a series of exercises for various aspects of TDDA, available on the blog, with follow-along screencasts.

This post is a standing post that we plan to try to keep up to date, describing options for obtaining the open-source Python TDDA library that we maintain.

Rexpy is a powerful tool we created that generates regular expressions from examples.

A recent post described the new ability to run a subset of ReferenceTest tests from the tdda library by tagging tests or test classes with the @tag decorator.

The test-driven data analysis library, tdda, has two main kinds of functionality support for testing complex analytical processes with unittest or pytest support for verifying data against constraints, and optionally for discovering such constraints from…

It is common, when working with tests for analytical processes, for test suites to take non-trivial amount of time to run.

Our previous post introduced the idea of data provenance (a.k.a. data lineage), which has been discussed on a couple of podcasts recently.

In Episode 49 of the Not So Standard Deviations podcast, the final segment (starting at 59:32) discusses data lineage, after Roger Peng listened to the September 3rd (2017) episode of another podcast, Linear Digressions, which discussed that subject.

We have a new White Paper available: Automatic Constraint Generation and Verification Abstract Correctness is a key problem at every stage of data science projects: completing an entire analysis without a serious error at some stage is surprisingly hard.

Bad data is widespread and pervasive.1 Only datasets and analytical processes that have been subject to rigorous and sustained quality assurance processes are typically capable of achieving low or zero error rates.

This post is a standing post that we plan to try to keep up to date, describing options for obtaining the open-source Python TDDA library that we maintain.

Last night I went to The Protectors of Data Scotland Meetup on the subject of Marketing and GDPR.

A quick-reference guide ("cheat sheet") is now available for the Python TDDA library.

Today we are announcing some enhancements to Rexpy, the tdda tool for finding regular expressions from examples.

Yesterday, email subscribers to the blog, and some RSS/casual viewers, will have seen a half-finished (in fact, abandoned) post that began to try to characterize success and failure on the crowd-funding platform Kickstarter.

It is a primary responsibility of analysts to present findings and data clearly, in ways to minimize the likelihood of misinterpretation.

We have written a 1-page summary of some of the core ideas in TDDA.

Rexpy Stats We previously added rexpy to the Python tdda module.

Since the last post, we have extended the reference test functionality in the Python tdda library.

Motivation There's a Skyscanner data feed we have been working with for a year or so.

Background We recently extended the tdda library to include support for automatic discovery of constraints from datasets, and for verification of datasets against constraints.