Cole Hoffer - AI Engineering Blog

Build continuous ratings from pairwise LLM judgments, without comparing every candidate pair on every task.

How human follow-up behavior reveals response quality, and why specific instruction adherence outperforms vague relevance scoring.

Using citation behavior in production RAG systems to generate labeled training data for domain-specific reranking models.

Using LLM-based classification as a second pass to filter retrieval candidates when similarity thresholds fail to generalize.

Using Hypothetical Document Embeddings to bridge vocabulary gaps between user queries and specialized document corpora.

Why decomposing queries into structured filters before semantic search improves retrieval precision and performance.

Combining chain-of-thought reasoning with logprob extraction improves LLM classification accuracy while giving you real confidence scores.