Reflective altruism

Like children opening their eyes to a new day, having gone to bed the previous night as tufts of snow began falling, we dash to the window and lift ourselves to the tips of our toes to behold a landscape transformed: a winter wonderland glittering with po…

There’s never just one cockroach in the kitchen.

There’s a temptation, when solving the world’s toughest and most-important problems, to throw money around.

Building on existing theoretical results demonstrating powerseeking incentives for most reward functions, we investigate how the training process affects powerseeking incentives and show that they are still likely to hold … In a setting where the trained…

If powerful artificial agents are coming, we want to ensure that they’re both shutdownable (they shut down when we want them to shut down) and useful (they otherwise pursue goals competently).

In experiments spanning more than 100,000 trials across thirteen large language models, we show that several state-of-the-art models presented with a simple task … sometimes actively subvert a shutdown mechanism in their environment to complete that task …

Perhaps one day we will create powerful artificial agents.

There is no falsification before the emergence of a better theory.

On your deathbed, God brings good news.

Subscribe to keep reading Become a paid subscriber to get access to the rest of this post and other exclusive content.