Mark Needham

I often need to minimise the schema and table config files that you use to configure Apache Pinot so that they don’t take up so much space.

Jeff Sackmann’s tennis_atp repository is one of the best collections of tennis data and I wanted to ingest the ATP Tour singles matches using the DuckDB CLI.

While in DuckDB land, I wanted to create a list of numbers, just like you can with Cypher’s range function.

I’ve been using the Arc Browser for a couple of months now and one of my favourite things is the simplicity of the plugin (or as they call it, 'Boost') functionality.

(Co-authored with Michael Hunger) Venkat Subramaniam is a legendary speaker on the tech conference circuit whose presentations are famous for executing arbitrary code snippets and showing the results as a tooltip directly in the editor.

I needed to crop a video that I used as part of a video on my YouTube channel, Learn Data With Mark, and Camtasia kept rendering a black screen.

Another gem from Fluent Python is that you can name slices.

I’ve been reading Fluent Python and learnt about pattern matching with the match/case statement, introduced in Python 3.10.

Working with dates in SQL can sometimes be a bit tricky, especially when you need to extract specific information, like the decade a date belongs to.

I’ve been playing around with the Citi Bike Stations dataset on Kaggle with DuckDB and ran into trouble when trying to convert a column containing epoch timestamps to a timestamp with timezone support.

In this blog post we’re going to learn how to build an application to compare the matches between two ATP tennis players.

I’ve been playing around with DuckDB over the weekend and ran into an interesting problem while using the Relational API in the Python package.

I’ve been working on a blog post showing how to use Geospatial indexes in Apache Pinot and ran into an interesting exception that I’ll explain in this blog post.

In this blog post we’re going to learn how to write a SQL query to join two tables where one of the tables has multiple rows for each key.

I’ve been playing around with Flink as part of a workshop that I’m doing at JFokus in a couple of weeks and I wanted to export some data from Flink to Apache Kafka in a nested format.

As part of a JFokus workshop that I’m working on I wanted to create a Flink table around a Kafka stream that I’d populated from MySQL with help from Debezium.

I was recently trying to convert a CSV file to Parquet format and came across a StackOverflow post that described a collection of different options.

I’ve recently been working with Debezium to get the Pizza Shop product catalogue from MySQL into Apache Kafka and ran into an issue when querying the resulting stream using kcat and jq.

I was recently working on a Kafka streams data generator, where I only wanted to publish events once the time on those events had been reached.

Writing blog posts that aim to go viral on social media is a well-known content strategy, but in this post, I want to persuade you that you should blog for Google as well.

I recently wrote a post on the StarTre blog describing the inserts from SQL feature that was added in Apache Pinot 0.11, and while writing it I came across some interesting exceptions due to configuration mistakes I’d made.

I recently wrote a post on the StarTre blog describing the inserts from SQL feature that was added in Apache Pinot 0.11, and while writing it I came across some interesting exceptions due to configuration mistakes I’d made.

I recently wrote a post on the StarTre blog describing the inserts from SQL feature that was added in Apache Pinot 0.11, and while writing it I came across some interesting exceptions due to configuration mistakes I’d made.

I’ve been playing around with DuckDB, the new hotness in the analytics space, over the last month, and my friend Michael Hunger asked whether you could use it to compute a diff of Apache Parquet schemas.

I was recently trying to ingest some JSON files into Apache Pinot from an S3 bucket and came across an exception when trying to pass a variable to the LaunchDataIngestionJob command I was using the following ingestion job specification: config/job-spec.yml

I was debugging something in the Apache Pinot code earlier this week and came across the FileSystems.getDefault().getPathMatcher function, which didn’t work quite how I expected.

We’re deploying the StarTree developer site, dev.startree.ai, to Vercel, and recently needed to do some redirects of a few pages.

I’ve been working on an Apache Pinot dataset where I ingested a JSON document stored in a CSV file.

I’ve been working with Docusaurus to build the dev.startree.ai website over the last few months and I wanted to add a custom page with a sidebar similar to the one that gets automatically generated on documentation pages.

As I mentioned in my last blog post, I’ve been working on an Apache Pinot recipe showing how to ingest data from S3 and after I’d got that working I moved onto using the SegmentGenerationAndPushTask to poll S3 and ingest files automatically.

I’ve been working on an Apache Pinot recipe showing how to ingest data from S3 and I needed to pass in my AWS credentials to the docker exec command that I was running.

As I mentioned in my blog post about building a Real-Time Crypto Dashboard, I’ve recently been working with the Dash low-code framework for building interactive data apps.

As I continue to build an Apache Pinot demo using CryptoWatch data, I found myself needing to optimise some queries so that the real-time dashboard would render more quickly.

I’m currently working on an Apache Pinot demo using data from Crypto Watch, in which I was using the lookup function and had a bug in my query that didn’t return the clearest error message.

In this blog post we’re going to learn how to work around a bug when trying to ingest CSV files with the same name into Apache Pinot.

Sometimes when I start up a local Pinot cluster after doing a hard shutdown (by restarting my computer) I noticed that the Pinot Data Explorer shows controllers, brokers, or servers in a bad state.

I was recently building a Streamlit app in which I was looping through a stream of values and wanted to only print out the most recent value.

I recently had a typo in a Pinot ingestion transformation function and wanted to have Pinot re-process the Kafka stream without having to restart all the things.

I’ve been doing more Apache Pinot documentation - this time covering the JSON functions - and I needed to quickly write some data into Kafka to test things out.

I’ve recently been learning all about Apache Pinot’s sorted forward indexes, and in my first blog post I explained how they work for offline tables.

I’ve recently been learning all about Apache Pinot’s sorted forward indexes.

I’ve been working with Strava data again recently and wanted to extract all the lat/long coordinates recorded for my runs.

Recently I’ve been playing around with geometry objects in WKT format while documenting Apache Pinot’s Geospatial functions.

One of the most common questions in the Apache Pinot community Slack is how to work out which indexes are defined on columns in Pinot segments.

In our last post about the Chicago Crimes dataset and Apache Pinot, we learnt how to use various indexes to filter columns by exact values.

In this post we’ll learn how to use the same Pinot segment in multiple tables.

In this post we’ll learn how to deal with a field that contains DateTime strings when importing a CSV file into Apache Pinot.

In Neha Pawar’s recent blog post, What Makes Apache Pinot fast?

I’ve been playing around with one of my favourite datasets from the Chicago Data Portal and spent a while figuring out how to import columns that contain spaces into Apache Pinot.

In my continued exploration of Apache Pinot, I wanted to spin up all the components individually rather than relying on one of the QuickStarts that takes care of that for me.