Mark Needham

DuckDB 1.2 is here, packed with new features to make SQL even more user-friendly.

ClickHouse recently added support for runtime provided parameters in views, so I wanted to try it when querying the MidJourney messages dataset.

I really like DuckDB’s PIVOT clause and along with some others wish that ClickHouse supported it too.

I’ve been playing around with Simon Willison’s llm library over the last week and I have to say I love it!

In this blog post, we’re going to learn how to send multiple prompts to vision models when using Ollama.

A couple of days ago OpenAI released a new version of Whisper, their audio to text model.

rerankers provides a unified API for various reranking models, including any that use transformers, FlashRank, RankGPT, RankLLM, and more.

DuckDB 1.1 was released a couple of weeks ago and there are a couple of features that make dynamic column selection even better.

One of my favourite things about DuckDB is the innovations it’s made in SQL.

I recently needed to search for an icon in the AWS [asset package](https://aws.amazon.com/architecture/icons/) and wanted to share a little script that I wrote.

We recently had a question on ClickHouse Community Slack about configuring the network_compression_method on an individual query basis and across all requests.

I’ve been playing around with different approaches for Retrieval Augmented Generation (RAG) recently and came across a blog post describing Reciprocal Rank Fusion, a hybrid search technique.

I’ve been learning about Hybrid Search via this blog post, which describes the Reciprocal Rank Fusion algorithm, and I wanted to implement and use it in a DuckDB query.

I’ve been playing around with ClickHouse’s Amazon reviews dataset and ran into an interesting problem when trying to set the allow_nullable_key setting.

Mistral AI recently released version 3 of their popular 7B model and this one is fine-tuned for function calling.

The recent 0.1.33 release of Ollama added experimental support for running multiple LLMs or the same LLM in parallel.

A tricky problem when deploying LLM-based chatbots is working out how to stop them from talking about topics that you don’t want them to talk about.

I’ve been playing around with the outlines library and needed to install llama.cpp as a result.

In the 0.10 version of DuckDB, a breaking change was made that stops implicit casting to VARCHAR during function binding.

One of my favourite tools in the LLM space is Ollama and if you want to learn how to use it, there’s no better place than Matt Williams' YouTube channel.

I’ve been playing around with the YouTube API to analyse comments on YouTube videos and needed to use pagination to get all the comments.

For quite some time I’ve been wondering how to get access to an environment variable in a ClickHouse Local and finally today I have a solution, which we’ll explore in this blog post.

I was recently working with a CSV file that contained a bunch of words and I wanted to render them on the console so that you could see all of them at once without any scrolling. i.e.

I was recently reading Simon Willison’s blog post about embedding algorithms in which he described how he’d used them to create a 'related posts' section on his blog post.

LLaVA (or Large Language and Vision Assistant), an open-source large multi-modal model, just released version 1.6.

This week Ollama released a Python/PyPi library to go with their awesome tool for running LLMs on your own machine.

When running queries with ClickHouse Local, the results are rendered back to the screen in a table format in blocks.

Retrieval Augmented Generation (RAG) is a technique used with Large Language Models (LLM) where you augment the prompt with data retrieved from a data store so that the LLM can generate a better answer to the question that is being asked.

After a few months of using ClickHouse, I’ve got quite used to using the SELECT <expr> EXCEPT modifier, which lets you remove columns based on a regular expression.

I’ve been playing around with NumPy data in ClickHouse this week and wanted to share what I learnt when checking for equality of float values.

I was playing around with neovim last week and despite installing it via Homebrew, ran into a weird permissions error.

I’ve been working with a dataset in ClickHouse where I compute currency values and I really struggled to figure out how to get numbers whose decimal part is divisible by 10 to have a fixed number of decimal places.

I recently came across insanely-fast-whisper, a CLI tool that you can use to transcribe audio files using OpenAI’s whisper-large-v3 model or other smaller models.

I needed to quickly generate some data to write to S3 for a recent video on the ClickHouse YouTube channel and it seemed like a good opportunity to try out ShadowTraffic.

I wanted to get two of my favourite tools in the LLM world - llmlite and llamafile - to play nicely and ran into an issue that I’ll explain in this blog post.

This week I’ve been making a video showing how to use ClickHouse’s S3Queue table engine, which allows streaming import of files in an S3 bucket.

Before a recent meetup talk that I did showing how to do analytics on your laptop with ClickHouse Local at Aiven’s Open Source Data Infrastructure Meetup, I needed to download a bunch of Parquet files from Hugging Face’s midjourney-messages dataset.

I’ve been playing around with the Mid Journey Parquet metadata that I wrote about in my last blog post and struggled quite a bit to get the query to do what I wanted.

I’m an avid reader of Simon Willison’s TIL blog and enjoyed a recent post showing how to sum the size of all the Midjourney images stored on Discord.

I was recently working with some very long arrays in ClickHouse and I wanted to select just a few values so that they didn’t take up the entire screen.

I was querying a ClickHouse server from a Python script a couple of days ago and ran into an error message when trying to create a Pandas DataFrame.

I’ve been working with ClickHouse’s access control/account management as part of a video that I created showing how to login to a ClickHouse server with an SSH key, but getting it all setup locally was a bit fiddly.

I’ve been working with dates in ClickHouse today and I wanted to convert some values into epoch seconds/milliseconds to use with another tool.

I’ve been working with some data that’s in CSV format but has tab-separated values in some columns.

I’ve been playing around with the CTransformers library recently and getting it installed via Poetry was a bit fiddly.

I’ve been toying with the idea of using iPython as the Python REPL for videos on @LearnDataWithMark, but I wanted to disable the autocomplete functionality as I find it too distracting.

I’ve run into an interesting error a few times when using the Poetry package manager over the last few weeks and wanted to document it in case anyone else has the same problem.

GGUF (GPT-Generated Unified Format) has emerged as the de facto standard file format for storing large language models for inference.

I’ve been playing with clickhouse-local again this week and ran into an interesting issue when persisting a table that I thought I’d document for future Mark.

I’ve been trying to install Apache Superset so that I can use it for a demo and ran into an issue with a secret key when trying to install it.