Nicholas Carlini

An opinionated perspective on how to do important research that makes a difference (and sometimes win awards).

Are the harms that LLMs have caused, and will soon cause, worth the benefits they may bring?

A feature-complete gate-level microcoded Intel 4004 in 4004 bytes of C, capable of emulating the original Busicom calculator ROM for which the chip was originally designed.

A small hardware description language implemented as a DSL on Python, with a small 170 LoC 32-bit RISC CPU.

Future LLMs have the potential to cause significant harm due to their ruthless effiency.

I have very wide error bars on the potential future of large language models, and I think you should too.

My work on privacy-preserving machine learning is often cited by lawyers arguing for or against how generative AI models violate copyright.

I have decided to leave Google, and will be joining Anthropic to continue my work on machine learning security

A one-year review of people's predictions on an AI-forecasting survey I made last year.

I wrote a (list of) regular expressions that will play a (not very good) chess game by running a 2-ply minimax search.

I let a language model write my bio.

You should forecast the future of AI in this quiz, so that you can see just how right or wrong you are.

I don't think that AI models (by which I mean: large language models) are over-hyped.

Yesterday I was forwarded a bunch of messages that Prof.

IEEE SP 2024 (one of the top computer security conferences) has, again, accepted an adversarial example defense paper that is broken with simple attacks.

A benchmark of ~100 tests for language models, collected from actual questions I've asked of language models in the last year.

How do I pick what research problems I want to solve?

So let's suppose you had a 1980s Apple ProFile Hard Drive, and you wanted to recover the data.

Building a chess bot that queries GPT-3.5-turbo-instruct to play chess at the level of a skilled human player.

TODO

This program is a dependency-free implementation of GPT-2, including

I recently got back from attending USENIX Security 2022, and someone pointed out to me that it's been five years since I wrote Towards Evaluating the Robustness of Neural Networks (with my at-the-time advisor) and they asked if I had any thoughts on this…

A brief discussion about a tool I use to make rapid iteration in ML research possible.

A recent paper ('A Roadmap for Big Model') has copied a bunch of text from over a dozen prior papers.

Abstract: Improving digital logic gates on Conway's game of life by allowing 8-bit logic gates instead of boolean logic gates.

This page contains a few checklists that help prevent embarrassing issues when releasing research papers online (e.g., via arXiv or a conference publication).

Abstract: An implementation of a minimal CPU on Conway's the Game of Life (an 'unlimited register machine'), and runs at ~10Hz.

Abstract: This post describes improvemnets made to my prior digital logic gate constructions (e.g., AND/OR/NOT) built on top of Conway's Game of Life, resulting in 100x faster simulations.

This year I entered in JS13K 2020, a game jam for JavaScript games in under 13KB (total size).

InstaHide (a recent method that claims to give a way to train neural networks while preserving training data privacy) was just awarded the 2nd place Bell Labs Prize (an award for finding solutions to some of the greatest challenges facing the information…

For the third year in a row, I participated in JS13k 2021, where you're tasked with making a game in 13kB of JavaScript.

I recently broke a defense to be published at CCS 2020, and this time I recorded my screen the entire time---all two hours of it.

Abstract: Using AND/OR/NOT gates built on top of Conway's Game of Life, this post walks through how to construct a actual circuits, for example a 7-segment display.

Abstract: This post walks through how to construct digital logic gates (AND/OR/NOT) on top of Conway's Game of Life, demonstrating its Turing completeness.

Abstract: We (again) broke a large collection of published defenses to adversarial examples.

This year I entered in JS13K 2019, which asks people to develop games in under 13K of JavaScript.

Late last year I decided it would be fun to build a 3D renderer in JavaScript.

Abstract: A continuously-updating list of all 1000+ papers posted to arXiv about adversarial examples.

Abstract: This reading list provides an introduction to the field of adversarial examples for machine learning models.

Abstract: This document contains a collection of advice for performing adversarial example defense evaluations.