Pablo Lopez - Software Engineer & AI Contractor | Edinburgh, UK

I load tested a Databricks SQL Warehouse as the backend for a high concurrency API, to find out whether you can skip the operational database.

This is the story of how I overengineered a software project because I made the wrong assumptions too early.

Over the last 3 years I’ve spent a lot of time building highly scalable Data APIs, to enable teams to access data from different sources with minimal effort.

Reflecting back to the time I was deciding whether to start my own blog site has made me remember that it wasn’t an easy decisions.

Over the past few months I’ve been using AI agents a lot to automate many engineering tasks that otherwise would take me much longer to do.

Step 1 — Target Audience The Reader A junior, mid, or senior data engineer who builds data pipelines in the warehouse (DBT, Databricks, Snowflake) and orchestrates them with Airflow.

Module 9: Ownership, security and new ways of working Closing section.

Resources There Is an AI Code Review Bubble - Honest take on why most AI code review tools are fundamentally limited.

Resources Claude Code: Agent Teams - Official documentation on running multi-agent teams with Claude Code.

Introduction At this point we have already seen how to use spec driven development to create a structured engineering workflow and using feedback loops te ensure the code is validated and meeting the necessary standards.

Introduction Once you get to the implementation phase, you can simply ask your agent to implement the task and end it there.

Module 6: CLIs, MCPs and debugging Hands-on: attendees connect a CLI or MCP to their agent and use it in a spec or debug session.

Module 3: Spec driven development First hands-on moment.

Introduction — what you will have set up by the end of this chapter IDE agents vs CLI agents — the two flavours, narrow scope to CLI for this chapter Understanding the workspace — mental model of what the agent can see, do, and access before you install a…

Module 1: How we got here Sets the stage.

Coding agents work best when they have a good feedback loop they can use to verify their changes.

Pablo Lopez Santori Edinburgh, UK | pablo.lopez.santori@gmail.com | github.com/wanderdust | wanderdust.github.io/newsletter Summary AI and Machine Learning Engineer with 6+ years of experience building production AI systems, from LLM-powered agents and RAG

If you were to peek inside Postgres and see how the tables are stored, you’d be surprised they look nothing like the outuput you get when you do a SELECT query.

This has been one of the years where I’ve learned the most things, probably since I first started learning programming almost 10 years ago.

The hardware Compute nodes (thinkpads) NAS (Optional) Setting up the nodes 2.1) Installing Ubuntu Server 2.2) Setting up a static IP on the nodes See which is your ethernet name by looking at ip addr.

I have passed the AWS Architect Professional exam with a score of 791/1000!

In this post I wanted to describe the different options if you are considering a data role.

Today I wanted to write about something that happens to me at work all the time: the process of asking for technical help, helps me discover the answer to my question.

Typical Academic Roles (Career Progression) Role Approx.

Slowly changing dimensions, what a confusing name.

Interviewing for other jobs in tech can make you a more competent engineer.

When working with distributed systems you can encounter terminology such as “replicas” or “standby” intstances.

Practical strategies to protect your serverless applications from runaway AWS bills, including caching, rate limiting, WAF rules, and DynamoDB safeguards.

GOAL: Highlight the differences between these strategies.

After ignoring the whole “agent” hype for a while, I started to get that inevitable feeling of being left behind.

I recently passed the AWS Solutions Architect Associate exam with a score of 848.

Introduction This is a guide to myself if I need to deploy an Appsync App in the future.

I recently had to run load testing for an API that fetches data from Postgres.

In this post, guest writer Billy interviews Pablo on learning Dvorak.

Replication refers to having copies of your data across different instances.

Fiction Orbital This one tells the story of a team of 5 austronauts in their space mission.

I currently have a basic setup in NeoVim: I use NvChad by default with treesitter to get pretty text highlighting, telescope to navigate files, conform for all my formatting needs and some python utilities such as pyright.

In this post, I share how to build a Reverse ETL pipeline to upsert data from Delta (Databricks) into Postgres to provide sub-second response times to our tables.

The other day my team came across a peculiar situation with our Postgres database.

I recently needed to run load tests on a PostgreSQL database.

We all know that person that always asks for unit tests on your Pull Request.

Indexes are like going to the airport and looking at the signs to see where your gate is located.

Learning To prepare for the exam I took the Udemy Kubernetes Certified Application Developer (CKAD) with Tests which I could access for free through my company.

This is my story of how I went from non-tech into becoming a Machine Learning Engineer.