Sebastian Raschka, PhD

A Look at Recurrent Depth, Hidden Chains of Thought, and Recent Research on Looping Transformer Blocks

A short note on OpenAI Astra, recurrent depth, looped transformers, Nanbeige 4.2, and the Mixture-of-Recursions paper.

A short video on how conventional LLMs relate to reasoning models and agents, along with Python and PyTorch setup using uv.

Two free live book club Q&A sessions on September 3 about Build a Reasoning Model From Scratch and related reader questions.

Architecture note on GLM-5.3-Flash, formerly Ox Alpha, covering its KDA and MLA/DSA hybrid attention, sparse MoE backbone, and four-stream mHC residual path.

A 48-minute video walkthrough of token sampling, watermark detection, and removal

An End-to-End Project With Dataset Construction, Model Training, Local Deployment, and RLVR

Short illustration of how Claude's text watermarking is supposed to work based on Anthropic's released materials.

Short note on the Amazon availability of Build a Reasoning Model From Scratch and a warning about counterfeit black-and-white copies sold through Amazon India.

Short architecture note on Meta Muse Glimmer 30B, including gated local and global GQA, KV-cache efficiency, and release-time benchmark comparisons.

Short note celebrating the LLMs-from-scratch repository passing 100,000 GitHub stars and summarizing its learning materials.

Short architecture note on Kimi K3, including LatentMoE, Kimi Delta Attention, Attention Residuals, NoPE, multimodality, and inference-efficiency choices.

Short note on the architectures of six new open-weight models, including Nanbeige 4.2, Laguna S 2.1, Motif-3-Beta, Solar Open 2, Antares 1B, and BTL-3.

Short correction note for the random seed in Listing 6.5 on page 198 of Build a Reasoning Model From Scratch.

How LLMs Learn Low-, Medium-, and High-Effort Reasoning Modes

Architecture and benchmark notes on Thinking Machines Lab's 975B Inkling MoE, including short convolutions, relative-position bias, training, and effort control.

Short note celebrating Ahead of AI reaching 200,000 subscribers.

Short note on how GPT 5.6 model and effort choices map onto training-time and inference-time scaling, producing 72 configurations.

Short note announcing the release of Build a Reasoning Model From Scratch and linking the publisher and Amazon pages.

Short note linking a new article on setting up local coding agents with open-weight models.

Using Open-Weight Models in Local Coding Harnesses as an Alternative to Claude Code and Codex Subscriptions

Short note on trying local open-weight LLMs across Qwen-Code, Codex, and Claude Code harnesses.

How GLM-5.2 uses IndexShare to reuse sparse-attention token selections across layers and reduce long-context indexer computation.

A closer look at VibeThinker-3B, a Qwen2.5-Coder-based model whose reported reasoning gains come from a detailed post-training pipeline.

Architecture and benchmark notes for North Mini Code, Cohere's 30B-A3B MoE trained for repository, terminal, and code-generation tasks.

A curated roundup of notable LLM research papers that came out this year

Architecture notes on Nemotron 3 Ultra, including its 108-layer hybrid stack, Latent MoE scaling, NVFP4 recipe, MTP, and inference results.

Technical notes on MiniMax M2, including full attention, fine-grained MoE routing, agent training data, speed rewards, and self-evolution.

Short note on a DeepSeek Sparse Attention from-scratch implementation added to the LLMs-from-scratch repository.

From Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs

Short note linking a talk on implementing LLM architectures from scratch and comparing new open-weight model implementations against references.

A learning-oriented workflow for understanding new open-weight model releases

How coding agents use tools, memory, and repo context to make LLMs work better in practice

Architecture and benchmark notes for Gemma 4 31B and 26B-A4B, including hybrid attention, long-context changes, and evaluation caveats.

Compare two LLM architectures side by side across attention, decoder type, layer recipe, model scale, context length, and KV-cache use.

From MHA and GQA to MLA, sparse attention, and hybrid architectures

Browse 93 LLM architecture diagrams with model fact sheets, source links, concept explainers, and a side-by-side comparison tool.

Nemotron 3 Super combines Mamba-2, Latent MoE, sparse GQA, and shared-weight MTP in a throughput-oriented 120B-A12B model.

A Round Up And Comparison of 10 Open-Weight LLM Releases in Spring 2026

I recently sat down with Lex Fridman and Nathan Lambert for a comprehensive 4.5 h interview to discuss the current state of progress of AI, and what the...

Inference scaling has become one of the most effective ways to improve answer quality and accuracy in deployed LLMs.

A 2025 review of large language models, from DeepSeek R1 and RLVR to inference-time scaling, benchmarks, architectures, and predictions for 2026.

A curated list of LLM research papers from July–December 2025, organized by reasoning models, inference-time scaling, architectures, training efficiency...

Two years ago, I posted a list of Hello World examples for machine learning and AI on social.

Similar to DeepSeek V3, the team released their new flagship model over a major US holiday weekend.

This short article compiles a few notes I previously shared when readers ask how to get the most out of my building large language model from scratch books...

After I shared my Big LLM Architecture Comparison a few months ago, which focused on the main transformer-based LLMs, I received a lot of questions with...

The DGX Spark for local LLM inferencing and fine-tuning was a pretty popular discussion topic recently.

Multiple-Choice Benchmarks, Verifiers, Leaderboards, and LLM Judges with Code Examples

Previously, I compared the most notable open-weight architectures of 2025 in The Big LLM Architecture Comparison.