AI-Powered Personalization
A working-draft technical book on building production recommender systems, with eleven of sixteen planned chapters and companion code published openly under the MIT licence.
AI-Powered Personalization: An Industry Guide to Building Recommender Systems at Scale is a technical book I am writing about how recommendation and personalization systems are actually built in production.
The premise
Most of the useful knowledge about recommender systems is scattered. Practitioners entering the field assemble their understanding from engineering blogs, papers, and conference talks, each written at a different level of abstraction and none of them covering the whole lifecycle. There are good books on machine learning systems in general, but they tend to stop short of the specifics: how retrieval differs from ranking, or what a value function does after a model has scored candidates.
The book is an attempt at a single connected narrative covering that ground, from the data pipelines underneath a recommender to the operational concerns on top of it.
Who it is for
Three readers, in order of how directly the book speaks to them: machine learning engineers and data scientists moving into recommendations or personalization from other domains; engineers already in the field who want more depth on specific stages; and engineering & product leaders who need architectural patterns and a view of what reusable investment looks like across a portfolio of personalization use cases.
The model-heavy chapters assumes you can already build supervised models and does not teach deep learning fundamentals.
Scope and current stage
Sixteen chapters are planned across four parts: foundations, the core multi-stage pipeline, representation learning, and production operations. Eleven are written and publicly available as drafts in Word and PDF form, through Chapter 11 on user embeddings. The remaining chapters, covering deployment, latency, monitoring, and experimentation, are in progress.
Each chapter is accompanied by runnable code where the topic warrants it, published for eight chapters so far, covering two-tower retrieval, gradient-boosted and neural ranking, multi-task architectures, value functions and diversity re-ranking, and sequence and graph approaches to embeddings. Coding is not the focus of the book, so chapters carry only enough code to convey the mechanism, with fuller tutorials in the repository.
I am the sole author: I write the chapters, build the diagrams, and maintain the companion code. The book is being released on GitHub as free chapters under the MIT licence, published as each one is finished, and those chapters sit alongside a research paper search engine on the companion site.
Gallery
Figure 4.1 from the published Chapter 4, tracing the synchronous serving path and showing why the offline warehouse sits outside it.
Full resolution : Diagram of a serving-time ranking pipeline: a request carrying user ID and context flows into a candidate generation service, then a feature service combining batched precomputed lookups from an online store with on-demand transformations, then schema validation and tensor assembly, then a ranking model producing ranked items, with an offline warehouse below marked as not in the synchronous serving path.
The Reusability Hierarchy, from the published Chapter 10 on item embeddings, which argues that shared embeddings are the layer with the best balance of impact and feasibility, and which places managed recommendation APIs at the apex as the buy-not-build extreme.
Full resolution : Pyramid diagram titled The Reusability Hierarchy with six tiers. From the widest base upward: Ideas and Design Patterns, giving collaborative filtering, two-stage architectures, and A/B testing as examples; Data and Feature Pipelines, with shared feature stores and unified interaction logs; Code and Libraries, naming TFRS, RecBole, LightFM, and internal frameworks; Embeddings and Representations, highlighted with a border, labelled as the focus of Chapters 10 and 11, and marked by an arrow reading This chapter; Full Models, covering foundation models and multi-task systems; and Managed APIs at the narrow top, naming Amazon Personalize and Vertex AI Search. The tiers narrow toward the top because the artifacts are rarer and more heavily packaged, not because they matter more.Deep dives
No published articles yet.