On November 10, 2026, Mercari's engineering blog described how it rebuilt, with AI at the center, the process that converts product data from outside businesses into the format of Mercari's 'product catalog'. Before, rules decided by people were implemented in a Go batch. The new approach moves the conversion rules into natural-language prompts, has an LLM do the conversion, and also hands the work of writing those prompts to an AI agent. Mercari writes that in the quarter it adopted this, 'with about the same headcount it integrated 6 categories, the number of records integrated was more than 40 times higher than before, and the effort to integrate one new category fell by about 55%.'
First, how to read the numbers. On an offline test of 1,000 Pokémon cards, the share of records with all fields correct was 69.6% for the rule-based system, and 85.2% for the production model (gpt-5.4-mini) using the prompt tuned by the agent. The test covers Pokémon cards only, and a record counts as correct only if all 15 fields match, which is a strict measure. The 40x and 55% figures compare the quarter of adoption with the quarter before it; they are not a controlled experiment (Mercari's statement). Below, we go through the confirmed structure in order.
What was replaced
- ▸Target: bringing outside data into the product catalog. The product catalog is master data holding one record per 'product itself'. Listings (individual items) attach to that product. For Pokémon cards, one card such as 'Pikachu ex' is one record with attributes such as card name, set, rarity and card number. Registering by hand one record at a time limits coverage, so Mercari brings in data from businesses that already hold product data
- ▸Why conversion is needed: outside data is built for that business. For example, card name, rarity and finish may be written together inside the product description text. Mercari's catalog keeps them as separate fields, so they must be extracted from the text. Other work includes normalizing notation, deciding whether to list an item in the catalog, and composing product names and descriptions
- ▸The old way: catalog specialists (staff who know the data and the domain) wrote the rules into a spec, and engineers implemented them in a Go batch. These were rules that search for strings and dictionary terms, such as 'SR' for rarity or 'bulk sale' for set sales
- ▸Two limits of the old way. Even one rarity can be written as 'SR' or 'Super Rare', so rules had to be added each time a new way of writing appeared, and items were missed until then. Changing a rule meant updating the spec, implementing, testing, reviewing and deploying, so specialists could not fix rules themselves. Each new category also meant redoing the work from scratch, so one category per quarter was the limit
The architecture the blog confirms (two components)
- ▸LLM Data Conversion (the conversion mechanism): one record of outside data and a conversion prompt are passed to an LLM. The output comes back as JSON that follows an output schema, which code maps to Mercari catalog attributes before registration. In production, only the conversion step of the existing batch that imports outside data was replaced. Most conversion rules are written in the prompt by meaning, such as 'do not list set sales', rather than as lists of patterns, and the LLM absorbs small notation differences
- ▸Prompts are Markdown files: they sit in the repository as Markdown and are used almost as they are in production conversion. So even after integration starts, specialists can fix a rule without going through engineers when an error or a new way of writing is found. Once the rules settle, the engineers' job is to register the prompt in the batch and add a little code
- ▸Prompt Generation Agent (the agent that writes prompts): a specialist asks Claude Code something like 'make a conversion prompt for One Piece cards', and it proceeds interactively through data investigation, prompt writing, evaluation on samples and improvement. It was built and distributed as Agent Skills (a bundle of instructions, reference material and scripts), not as a new application. To improve the procedure, you only rewrite the instructions
- ▸The agent's five steps: (1) confirm the category, the provider and existing materials. (2) Investigate outside data with SQL on BigQuery (missing rates, value distributions, notation differences, exceptions such as graded items and set sales), organize the correspondence with Mercari attributes, and have the specialist confirm it. (3) Write the prompt and output schema. (4) Run it locally on hundreds of records and evaluate. (5) If there are problems return to (3); output the version that passes as final. The blog says a check that people spent days on for data with hundreds of columns now takes tens of minutes in dialogue with the agent
How LLM errors are kept down
- ▸1. Lock the output shape. The call uses Structured Outputs (a feature where you pass the fields and their allowed values as a JSON Schema and the model returns only JSON in that shape). If rarity is a field chosen from fixed values such as 'SR', unplanned spellings like 'Super Rare' and missing fields cannot occur. However, the blog also says this only locks the shape; picking a wrong value from among the allowed ones is not prevented
- ▸2. Separate what the LLM handles from what it does not. The LLM handles most of the conversion, but processing that is simple enough that the LLM's advantage is small, and processing where output variation would break data consistency, is deliberately written in code. One example is the check that a product already in Mercari's catalog is not registered again. If that wobbled, two catalog records for the same product would appear, so it is done in code and guaranteed by tests. The split is decided by converting real samples locally and looking at how much the output varies
- ▸3. An operations loop to find and fix errors. Registered data is checked regularly to find errors, which are first fixed by hand on the spot. But the batch converts again every time it imports outside data, so a hand fix alone would revert at the next update. So the fixed data is flagged and excluded from batch updates, and the prompt that caused the error is fixed. The fixed prompt is checked with a dry-run, which converts with the production code but writes the results to a CSV instead of registering, to confirm that the error is fixed and previously correct data has not broken. Once the fix is in production, the flag is removed and the data returns to updates
The numbers we can confirm (Mercari's statements)
- ▸Offline test setup: Pokémon cards. Mercari took 1,000 pairs of outside data and the Mercari catalog as the test set, and the agent did not see the test data while tuning, only development data. Scoring was exact match on 15 fields including card name, set, rarity and card number, and the main metric is the share of records with all fields correct (the all-fields rate). The agent that writes and tunes prompts ran on Claude Code, and the model is stated as Claude Opus 5.5
- ▸Rule-based all-fields rate: 69.6%.
- ▸gpt-5.4-mini, used in production: a prompt of only a few examples (few-shot only) scored 66.2%, below the rule-based system. The prompt tuned by the agent scored 85.2%, which is 15.6 points above the rule-based system. The blog says tuning added rules such as 'do not fill in values that are not in the input', which stops the model from filling in values from its own knowledge, and improved how options are ordered
- ▸gpt-5.6, a stronger model: 92.3% with few-shot only and 98.7% after tuning, 29.1 points above the rule-based system. The blog says the production model is gpt-5.4-mini, a choice that also considers cost
- ▸Results in the quarter of adoption (compared with the quarter before): before, one category per quarter was the limit; with about the same headcount it integrated 6 categories, and the number of records integrated was more than 40 times the previous level. The effort to integrate one new category fell by about 55%. In particular, checking the contents of outside data and building the conversion fell to one third or less of before. LLM usage fees are new, but the cost per record is very small and judged far smaller than the effort of hand-building rules (no amount is given)
How to read the numbers (our view)
- ▸'85.2%' is for 1,000 Pokémon cards and the measure that requires all 15 fields to match. It is not the accuracy for other categories or for all the data in operation. A single wrong field makes a record wrong, so the blog notes it comes out lower than per-field accuracy
- ▸'40 times or more' is, in the blog's wording, the number of 'records integrated'. The blog text did not let us confirm what is counted (catalog records or records of outside data). It is a separate figure from the number of categories (1 to 6)
- ▸'About 55% less' is the effort per new category. It compares the quarter of adoption with the quarter before, and is not a controlled experiment
- ▸'98.7%' is for a model not used in production. Do not mix it with the production gpt-5.4-mini's 85.2%. Even with the same few-shot prompt, gpt-5.4-mini scored 66.2% and gpt-5.6 scored 92.3%; the gap by model generation is large and tuning does not close it, the blog says
- ▸Model names are as the blog gives them. We did not check each model's specifications or availability
What still depends on people
Mercari itself writes that the loop for finding and fixing errors 'still leaves much to human hands.' People check data regularly to find errors, and putting a prompt fix into production needs a release. The more categories are integrated, the more this becomes a bottleneck, it says. As the next step, it plans to find errors by automated evaluation, have the agent draft prompt fixes and reflect them in production without a deploy, and shift specialists' role toward reviewing and approving the drafts. This is a plan; the blog does not say it is already running.
What we could not confirm
- ▸The breakdown of the 6 categories and the names of the partner businesses. The blog does not give them. The test used Pokémon cards, and One Piece cards appear as an example prompt request, and that is as far as it goes
- ▸The actual count behind '40 times or more'. The counts before and after are not given
- ▸The amount of LLM usage fees. The blog says only that the cost per record is very small
- ▸The real error rate after release. The accuracy figures in the blog are from the offline test
- ▸How much search or listing experience improved. The blog gives a general explanation that catalogs reduce search filtering and listing effort, but no figures for this initiative
- ▸The contents of the blog's Figures 1-4 and Tables 1-4. The text we could retrieve did not include the figures' and tables' contents as characters (they appear to be images), so we wrote from what the prose lets us confirm. For Tables 3 and 4 we used only the numbers written in the text
What to take from this case (our view)
- ▸Work where rules multiply with every way of writing suits prompts written by meaning. Even so, decide in advance what the LLM will not handle. Mercari kept processing that breaks consistency when it wobbles, such as the check that prevents duplicate registration, in code
- ▸Lock the output shape first. If the allowed values are fixed in a JSON Schema, unplanned spellings cannot come out. But mix-ups among the allowed values are not prevented, so you need a way to find them in operation
- ▸Do not stop at a hand fix. The order of flagging fixed data out of updates so it does not revert, fixing the prompt behind the cause, and returning it only after a dry-run is a useful reference for operating LLM-based conversion
- ▸Automate prompt writing, with evaluation built in. The agent investigates the data and evaluates on hundreds of records before fixing. Keeping the test data hidden during tuning is a precondition for trusting the numbers
- ▸Read numbers together with the evaluation conditions. Before saying 'accuracy rose', check which data, which metric and which model