www.marginalia.nu on marginalia.nu

As a general observation, I tend to be more productive when I know what to do next at any given moment.

This is a mostly technical release.

So the search engine is moving to a new server soon, thanks to the generous grant mentioned recently.

I ran into a bit of a puzzling situation yesterday, testing some of the new index construction changes before they’re going live in a few days.

I’m happy to announce that the generous people at FUTO have granted the project $15,000 with no strings attached to help the search engine out with some more server power.

Time: 1 hour of mostly waiting for the cucumbers to strain Ingredients Servings, up to 4 on paper, probably more 2 in practice because this is delish.

So… I’ve had the most unreal week of coding.

I’ve started going on a long walk each morning immediately as I wake up, and it’s had the unexpected side-effect of fixing my broken circadian rhythm.

This release mainly aims to improve the operational side of the search engine, with an emphasis of automating tedious manual processes and optimizing crawling and data processing to use fewer resources.

This is a bit of an what I’ve been working on style of post.

I recently got a smaller computer screen.

New Features Generator keywords To provide additional ways of selecting search results, a synthetic keyword has been added for the <meta name="generator" content="

I’m working on Marginalia Search full time.

This is a theory that’s previously been stated in log/39-normie-hypothesis.gmi, but I think it’s worth expanding on as it’s become very relevant with the recent Reddit shit-show actualizing just how bad that website has gotten along with social media in g…

This is primarily a bugfix release that primarily addresses some issues with a metadata corruption that was introduced in the previous release.

I use print debugging all the time I know how to use a debugger.

I killed the old memex.marginalia.nu site.

The computer in question is a model 15-df0002no bought in 2020.

This is the most basic-ass tomato sauce.

There is an episode of Star Trek where a character is for plot reasons trapped in a shrinking parallel universe.

I’ve come to think LLMs/GPTs/whatever are a threat to conventional search engines because the modern web is an unbelievably annoying dumpster fire.

Upgrading to 2023.03.1.

This information is outdated The marginalia search project info now lives on about.marginalia-search.com.

Loading the March crawl batch and upgrading to 2023.03.0.

I’ve moved Marginalia’s sources to Github.

This is a bit of a follow up to the previous post.

This information is outdated The marginalia search project info now lives on about.marginalia-search.com.

In general I don’t like to fuss over code, but this is exactly what I’ve been doing in preparation of the NLnet funded work.

No time like the project’s two year anniversary to drop this particular bomb… Marginalia’s gotten an NLNet grant.

This is a very brief post announcing a fascinating discovery.

A brief list of video games I’ve enjoyed the last few years.

I don’t know if I’m just imagining it, but has the Internet gone progressively more crazy the last decade or so?

For clarification, this is discussing no other thing called Memex than memex.marginalia.nu, the website you’re probably visiting right now.

Loading the December crawl batch.

The most common (and most costly) operation of the marginalia search engine’s index is something like given a set of documents containing one keyword, find each documents containing another keyword.

This is a write-up about an experiment from a few months ago, in how to find websites that are similar to each other.

While this post is about programming, it also draws an extended analogy to dungeons and dragons, specifically two of its classes, that correspond to two attitudes toward programming.

This information is outdated The marginalia search project info now lives on about.marginalia-search.com.

I get my best ideas when I’m not working.

This information is outdated The marginalia search project info now lives on about.marginalia-search.com.

One of the more common feature requests I’ve gotten for Marginalia Search is the ability to search by date.

By which I mean there are deeply problematic assumptions in the very notion of scaling: Scaling changes the rules, and scaling problems exist in both directions.

A very brief note to announce reaching a long term goal and major milestone for marginalia search.

A sustained DDoS attack brought the server down for about a day.

This information is outdated The marginalia search project info now lives on about.marginalia-search.com.

The following is a list of curiosities I’ve found while crawling the internet looking for websites.

Loading the August crawl batch.

Loading the August crawl batch.

This information is outdated The marginalia search project info now lives on about.marginalia-search.com.

In the primordial days of Marginalia Search, it used a dynamic approach to crawling the Internet.