Creează un cont pentru a te abona la acest blog și a-i urmări articolele pe pagina Lectură.
-
fewbottles.com is a working cocktail app that costs me a domain name and nothing else.
-
A Chief Engineer with over twenty-five years of experience in software architecture, development, and digital transformation.
-
I have been assembling data platforms for clients for over twenty years.
-
An analyst logs in, Claude calls trino_query, and Trino has to authenticate as someone.
-
If you operate the MCP server, you pre-register the clients.
-
A REST API with thousands of endpoints will generate thousands of functions for an agent if you run the usual OpenAPI generator.
-
This is MCP by Design, an ongoing development log of mcp-data-platform, a Go server I run in front of a warehouse, a catalog, object storage, APIs, other MCP servers, and a knowledge base, also hosted as Plexara.
-
A fifty-megabyte CSV from a vendor is sitting in a bucket, or about to be.
-
Agents write good reports and dashboards.
-
Function-level lint cannot see a god package. gocyclo at 10, funlen at 80, revive’s argument limits: all of those look inside one function.
-
I work from Los Angeles, at Deasil Works.
-
mcp-data-platform exposes knowledge as wiki-style pages.
-
Agents on mcp-data-platform write notes during a session.
-
The clearest reason to put your own hardware in the hybrid cluster is the GPU.
-
The platform so far runs on a set of cloud virtual machines, and that is the right default.
-
The 2020 book had a chapter on Blockchain.
-
Some work on a platform is a small piece of logic that should run on demand and cost nothing when idle: enrich a record, transform a payload, call out to a model, respond to an event.
-
Postgres is the platform’s default database, and most of the time it is the right answer.
-
A common cloud pattern is a file landing in a bucket that automatically kicks off some work: a CSV arrives and gets validated, an image gets a thumbnail, a log file gets compressed.
-
The two benchmark reports I published in July, the knowledge-layer ablation and the stored-knowledge study, both came out of the same piece of software: a benchmark harness in the bench/ directory of mcp-data-platform, roughly 36,000 lines of Go in its own
-
The platform stores, moves, and serves a lot of data, and the people working with it need a place to write Python, run a query, build a model, and explore.
-
A data platform that takes in device and sensor data needs a way to receive it that suits how those devices actually work: small, intermittent, often on flaky networks, sometimes a microcontroller with kilobytes of memory.
-
Every component in this series has been exposing metrics: Postgres, Kafka, OpenSearch, Cilium, the cluster itself, each one had a line about the monitoring stack scraping it “later.”
-
Agents running on mcp-data-platform do something the design never asked for: they store world state as knowledge.
-
When you built the cluster, one cilium install gave it pod networking, replaced kube-proxy, and encrypted traffic between nodes, three jobs the 2020 edition did with three separate pieces.
-
When you built the cluster, the container runtime under the kubelet was containerd, with Docker nowhere in sight.
-
Give a frontier model raw SQL access to a warehouse and ask it for last quarter’s top account by revenue.
-
The 2020 book covered building software on the platform, and it stopped there.
-
The Forgejo pipelines from the last post build container images.
-
A platform needs a place to keep its code, run its pipelines, and store the images it builds.
-
A frontier model can run SQL against everything you have built.
-
The platform now holds data in a lot of places: tables in Postgres, topics in Kafka, indices in OpenSearch, and Iceberg tables in the lakehouse.
-
Everything built so far stores, moves, and queries data.
-
A platform spends most of its life moving data between systems: pulling from an API, reshaping a payload, draining a queue into a table, fanning one feed out to three destinations.
-
The platform is about to grow tools that people log into: a dataflow canvas, a BI front end, dashboards.
-
A platform’s analytical questions, the roll-ups, joins, and slices over years of data, want a warehouse.
-
A data platform needs object storage: the S3-compatible bucket layer that holds a data lake’s raw files, the destination for database and stream backups, and the storage under the lakehouse tables coming next.
-
Some questions are slow on a relational database no matter how you index them.
-
When a platform needs to move events in real time, many producers, many consumers, durable and replayable, Kafka is the backbone.
-
I gave an agent one brief and pointed it at my own platform: connect as a real MCP client, exercise every tool, and find every place where what the tool says and what the tool does disagree.
-
Every platform needs a database, and for most of what you build it should be PostgreSQL.
-
Halfway through a query session an analyst types a sentence that is worth more than the query: “timestamps before March 2024 are UTC, after that they are America/Chicago.”
-
The 2020 edition exposed services to the outside world with the Nginx Ingress controller, a daemon on every node terminating ports 80 and 443.
-
Agents skip the steps you want them to take.
-
The cluster from the last post can schedule containers, but it cannot yet keep anything.
-
The standard way to give an agent access to three systems is to run three MCP servers.
-
Every platform in this series sits on one foundation: a Kubernetes cluster you own outright.
-
An MCP server is written by an agent, used by an agent, and now often evaluated by an agent, and every one of them can confabulate.
-
Ninety percent of what I search for now lands me on an article that was clearly generated, never read by a human, and written from nothing.
-
In 2020, Apress published my book, Advanced Platform Development with Kubernetes.