Stories from a Software Tester

The first and second posts in this series were about one person occupying two roles on the same project, sometimes switching between them within a single bit of work.

This is a follow-on to my previous post about what it looks like when a developer tests.

I was going to call this post “Interrogating My Own Code” but that was a bit opaque, even though it’s entirely accurate.

Awhile back I posted my call to action posts: Testers: Act Like a Developer.

I originally wanted to title this post “What Is It Like to Be Nothing?”

In my previous post, I talked about the cautionary aspect of AI hallucinating its own hallucinations.

I recently had an ongoing series on AI and Testing.

In two earlier posts I traced the path from Aristotelian to Galilean thinking as a way of understanding how testing developed as a discipline: how competing models of quality, and the slow maturation of experimental method, gave rise to something we might…

Previously I had talked about the idea of personal marketability when it came to learning AI.

In the previous post we looked at the idea of building up a pipeline with DSPy.

In the first post, we got everything set up to start exploring DSPy.

In my AI and Testing series, which ran for a couple of months, I focused heavily on the testing side of things.

This series has now covered eight metrics across two evaluation paradigms.

In the previous posts in the DeepEval series, we built up a diagnostic framework for evaluating RAG systems, covering Faithfulness, Contextual Precision, Contextual Recall, Contextual Relevancy, and G-Eval.

In the previous posts we looked at the Faithfulness and Contextual Precision metrics with DeepEval, and started building an intuition for how retrieval failures cascade into generation failures.

In my previous posts, I’ve been talking a lot about AI technology and tooling, and any enthusiasm within those posts came from helping people see how and where to test AI, and keep it explainable, interpretable, testable and, thus, ultimately trustable.

If you’ve been following my recent posts on how to test AI, you know that evaluating Large Language Models (LLMs) requires an entirely different mindset than traditional software testing.

In the previous post, we built a formal ontology from the Z-Machine specification, used it to drive code generation, and then turned it into a test oracle.

In the previous post, we looked at setting up an ontology based on a Z-Machine specification.

In the previous posts we looked at setting up a graph pipeline and auditing that pipeline.