On 9 October 2026 the Pinterest Engineering Blog described how Pinterest infers what a user is trying to accomplish. A fine-tuned 4-billion-parameter (4B) model reads an activity log and writes names for the user's goals, which the post calls "journeys", such as "Modern Farmhouse Kitchen" or "Raised Bed Vegetable Garden". The results feed email and push notifications.
First, what the post does not say. Its main result is a 1.1% lift in email click-through and a 1.3% lift in push opens over the old system, in an experiment in the US and Canada. The post does not say whether 1.1% and 1.3% are relative lifts or changes in the rate itself. Gains in daily, weekly and monthly active users are reported only as increases, with no figures, and there is no cost figure or share of journeys judged correct. This article separates what the post lets us confirm about the architecture from how to read those numbers.
What Pinterest handed to the AI
The task is inferring goals, to choose which notifications and recommendations to send. The post defines a journey as the intersection of a user's interests, intent and context at a point in time, spanning weeks or months and many sessions. "Summer dresses" is an interest; "figuring out what's in style for a summer wedding I'm attending in August" is a journey. One person may have five at once, such as a kitchen remodel, a vegetable garden and learning sourdough.
The aim is to follow the project someone is trying to finish, not the one Pin they just clicked. If the system can tell whether cabinet, countertop and backsplash Pins are one kitchen remodel, separate projects or noise, it changes what to retrieve, what to write in a notification, and when to stay quiet. The output is not a taxonomy label but a short, specific, ranked list of names the person would recognize.
The old clustering pipeline, and where it failed
- ▸The first system was a chain of stages. It extracted keywords from search queries and Pin annotations, clustered them hierarchically (each cluster a journey candidate), named each cluster by its top keyword, ranked and diversified the clusters, and sorted them by rules into evergreen or situational and ongoing or ended. It started from a few hundred human examples
- ▸It still worked. Against interest-based notifications, it lifted email click rate by 88%, push open rate by 32% and positive survey feedback by 23%
- ▸But errors compounded. The post lists three: fragmentation (cabinets, countertops and backsplash split into three journeys), generic buckets such as "Art", "Recetas" and "Shopping" when a cluster was messy, and incidental noise where a few meme or celebrity clicks took a top slot
- ▸It also could not write in other languages. It named journeys from an English annotation vocabulary, so it could not produce names in Spanish, for example
The architecture the post confirms (four parts and one flow)
- ▸Input (the prompt): a semi-static profile (gender, language, country, age range), the user's board names and a chronological activity log. Actions are reduced to a small vocabulary of search, save, like, click, comment and dislike, with search and save strongest and dislike included as evidence against. Events are grouped by date and action to save tokens, and the model reads a 360-day lookback of up to 150 events
- ▸Generation model: a fine-tuned 4-billion-parameter model (Qwen3-based) returns journey names as strict JSON in one call. Temperature 0, reasoning tokens off and at most 300 output tokens, so the same log gives the same reading. It outputs up to about twelve journeys of 1 to 5 words each, in the user's language
- ▸Post-processing: non-strings are dropped, duplicates removed case-insensitively and the first occurrence kept. The post says output is parsed defensively, because an occasional prose reply should not drop a user's whole personalization payload
- ▸Storage and use: a ranked list per user is stored and read when a notification is sent. Each journey name becomes a retrieval query; embedding-based retrieval pulls Pins, which existing Pin ranking models order. Journeys recently notified about are fatigued, and ones the user rejected are suppressed. Only then does the notification framework decide whether to send, and on which channel
- ▸The old system stays as a fallback: the post calls clustering the right cold-start system, and says it still exists as a fallback. Production inference is LLM-first
What the post stresses is collapsing a staged pipeline into one act of generation. The separate models for keyword extraction, embedding, clustering and naming are gone; one model reads the log and writes the answer. The trade-off is that policy once held in model weights and thresholds now has to be stated in the prompt: the ranking order (search and save above comment, like, click and dislike), "don't merge unrelated topics", "drop one-off outliers", and "don't invent unsupported journeys; an empty output is allowed". The post says Pinterest treats prompt changes with the same review discipline as model code, as launches.
From a large model to a small one (distillation)
- ▸A large model is the teacher. Frontier models can infer coherent journeys without task-specific training, but running them regularly for hundreds of millions of people is too expensive. So they generated synthetic examples, each activity history paired with a high-quality target journey. Teacher output improved when the model was allowed extra reasoning tokens before its final answer. The post does not name the teacher model
- ▸The students are Qwen3. Sizes of 0.6B, 1.7B, 4B and 8B were fine-tuned with SFT (supervised fine-tuning) and LoRA (updating small adapters instead of every parameter)
- ▸More examples stopped helping. Quality rose quickly over the first few thousand examples, then largely flattened. Selecting diverse, high-quality examples, especially difficult consolidation and multilingual cases, was worth more than adding similar ones
- ▸Why 4B. The 0.6B and 1.7B students were faster but less relevant and consistent on harder cases. Of the sizes tested, 4B offered the best balance of quality and throughput, so it became the production student
Serving at scale, and the safety lines
- ▸Scale: scoring hundreds of millions of people is a batch problem with online-shaped latency. A full refresh runs when the algorithm changes, and a daily incremental pass covers recently active users
- ▸Serving stack: NVIDIA Dynamo in front of vLLM decode workers on L40S GPUs, with PagedAttention, continuous batching and prefix caching. Online serving beat the offline workflow on throughput. In production a cluster of about 100 GPUs sustains about 775 requests per second, with a median of about 1.2 seconds and a p95 of about 3.0 seconds
- ▸The biggest saving was not a GPU. Pinterest records when each user was last inferred and skips anyone processed in the last couple of days. Journeys do not change hour to hour, and re-deriving an unchanged journey is the most expensive kind of waste, the post says
- ▸On failure: a failed decode is retried for that user, and a partial list is never stored. An empty but valid JSON list counts as success. The post would rather send nothing journey-specific than a generic "Art"
- ▸No inference of sensitive traits: input keywords pass the notification content filter before reaching the prompt, and output names pass that filter and generative content safety filters. The model is told never to infer or target topics related to health conditions, politics, religion, sexual orientation, gender identity or race
- ▸Minimized inputs and exclusions: ages are bucketed, because the model needs a rough life stage rather than a birthday. Under-18 users in the EEA, Switzerland and the UK are not yet eligible, due to local laws
How they measured it, and the numbers
There is no ground-truth label for a user's latent goal. The post describes three levels of evaluation, from inspection to a causal test.
- ▸Same-user side-by-side comparison: clustering output and LLM output are put next to each other on the same user activity. Pinterest keeps the strong cases and the honest misses: over-generalizing to "Home Cooking" when the log is clearly "Spam Kimchi Fried Rice", and dropping a well-supported side interest
- ▸Evidence-grounded diagnosis: a separate language model scores emitted journeys on relevance to the visible context, support from the profile and activity log, diversity and granularity. Support is the important axis: a fluent name that is not in the log is a failure, even if the user might have clicked it
- ▸Online experiment: with the old clustering system as control, the first ramp to US and Canada users showed lifts of 1.1% on email click-through and 1.3% on push opens, plus several statistically significant sitewide engagement and retention gains. A later multilingual international rollout replicated the direction, and shipping both experiments increased daily, weekly and monthly active users
How to read the numbers (our view)
- ▸88%, 32% and 23% belong to the old pipeline, not the new model. They compare it with interest-based notifications. The LLM approach was measured in a separate experiment against the old pipeline
- ▸1.1% and 1.3% are differences from the old pipeline. The post writes "lifts of 1.1% and 1.3%" and does not say whether they are relative lifts or changes in the rate itself. Do not read them as "the click rate rose by 1.1 percentage points"
- ▸"Statistically significant" and "increased active users" come with no figures. The size of the gain cannot be confirmed
- ▸About 100 GPUs and about 775 requests per second is Pinterest's production figure. It is not what any model or scale will reach; it is conditional on this model type running on L40S GPUs
- ▸"A few thousand examples" is about training data. The post does not say it applies to other tasks
Still in experiment (not the production default)
The post says everything in this section is under experiment, trained or rolling out, and not what every user gets today. We list only what it states.
- ▸Semantic IDs: each Pin is compressed by an RQ-VAE into five levels of 11 bits instead of a bag of tags, shortening the input. Continued pre-training maps the model's knowledge onto the catalog, with SFT on top. The cost is interpretability: a person cannot read an ID prompt mid-debugging. Users with fewer than ten such events fall back to tags
- ▸Offsite conversions: page visits, add-to-carts, sign-ups and checkouts are added as a compact, safety-filtered summary when onsite activity is sparse. Onsite evidence outweighs offsite, and offsite-only journeys are not ranked at the top
- ▸One model, several outputs: each journey can come with deeper cuts and adjacent topics, and the same model writes notification titles. Journey names are finalized first, and a name is never reshaped to make its copy sound better
- ▸Learning from user feedback: the "Are you interested in <journey>?" Yes and No buttons in emails, and saves, clicks and hides in notification sessions, feed preference optimization. Offline results are promising and online A/B tests are running; no figures yet
What we could not confirm
- ▸Whether 1.1% and 1.3% are relative or absolute, and the size of the sitewide and active-user gains. The post gives no figures
- ▸The teacher model's name, the number of training examples and the training cost. Not stated
- ▸The share of journeys judged correct (evaluator scores or human ratings). The three evaluation levels are described, but no result figures are given
- ▸Language coverage beyond the Spanish example. There is no list of supported languages or per-language results
- ▸The post's figures and tables as images. They are images, so this article relies on the text and captions
What to take from this case (our view)
- ▸Before chaining more models, try one generation. If small errors pile up at each stage (fragmentation, generic names, noise), collapsing the pipeline into one generation is an option. Policy then moves into the prompt, so Pinterest's choice to review and launch prompts like model code is worth copying
- ▸The old method can stay as a fallback. Clustering did its job as the first system and is still the fallback, a footing if the replacement fails
- ▸A large model teaches; a small model serves. To avoid running a large model every time, train a small one on examples the large one wrote. The quality of the examples, especially hard and multilingual cases, mattered more than their number, which says where to spend effort when collecting data
- ▸Treat doing nothing as a success. With no supported goal, return an empty list rather than inventing one, and count a name that is not in the log as a failure
- ▸Read numbers with their definition and comparison. "A 1.1% lift" alone is not something to adopt as your own target