With 30M residual-stream activations on disk, I trained a TopK sparse autoencoder on gpt2-xl's layer 7, read off what its features fire on, and then closed the loop the way interpretability is supposed to: I picked a feature by what it correlates with, clamped it on during generation, and watched the model produce exactly that. The feature I picked turned out to be the Unicode replacement character.

Before you can train a sparse autoencoder you need a few tens of millions of residual-stream activations on disk. Building that pipeline turned into a small tour of practical GPU performance: a roofline that says 'stop at batch 32', a writer rewrite that doubled throughput, and torch.profiler quietly answering every question I was going to point nsys at.

Went from a toy 2D spiral DDPM to a 130M-parameter class-conditional Diffusion Transformer on full ImageNet-1k, found an 8.3x training speedup with TF32 + bf16 + torch.compile, completed the paper's full 400K-step schedule, then scaled to the 675.8M-parameter DiT-XL/2 on 2 GPUs — where a late-training divergence turned into a lesson about gradient clipping and checkpoint hygiene.

How language models are turned from raw next-token predictors into assistants — the three post-training stages (SFT, preference optimization, specialized/chat fine-tuning) — grounded in a worked example: I ran the whole pipeline on a 124M GPT-2 I trained from scratch. Includes real base→SFT→DPO→chat interactions and an in-browser demo you can try.

I took the 124M GPT-2 I trained from scratch and ran the canonical post-training pipeline on it — supervised fine-tuning on Alpaca, then DPO on UltraFeedback, plus a multi-turn chat variant. SFT gave a dramatic jump to instruction-following; DPO made it 'sound' more like an assistant while demonstrating the alignment tax in miniature. There's an in-browser demo you can chat with.

Qwen-Image-Edit-2511 feeds a reference-image's VAE-encoded tokens through all 60 joint-attention blocks at every denoising step, discarding their own output every time. A per-block cosine-similarity study shows they're far more stable than they need to be recomputed from scratch — and a KV cache built on that finding gets a validated 1.3-1.7x speedup on 100 real edit tasks.

A minimal from-scratch DDPM trained on a 2D spiral, animated to show random noise being denoised back onto the data manifold step by step — plus a side-by-side comparison of stochastic vs. deterministic sampling.

I trained a 124M GPT-2 from scratch on 10B tokens of FineWeb-Edu on a single H100, benchmarked it against the real GPT-2, and traced how its capabilities — including an induction head — emerged over training. Notes on making the training practical, what the data choice did to the model, and where a from-scratch repro stops feeling like a demo.

I replicated Turner et al.'s and Soligo et al.'s emergent misalignment results on 2x H100s: a single rank-1 LoRA adapter on one MLP layer, trained only on bad medical advice, induces broadly misaligned behavior on unrelated topics. Then extracted the mean-diff misalignment direction, steered the base model with it, and tried ablating it from the EM model — replicating the paper's near-orthogonality finding but getting a result that flatly contradicts its ablation claim.

Accepted to ICLR 2026. I co-authored a post on LessWrong on removing LayerNorm from transformers by fine-tuning and implications for mechanistic interpretability (direct logit attribution, attribution patching, entropy neurons).

Co-authored with Aaron Kaufman for the ARENA capstone: we replicated MELBO (Mechanistically Eliciting Latent Behaviors) on Llama-3.2-1b-Instruct and did a hyperparameter sweep for the R value using diversity and coherence metrics.