Data quality & master data
Domain Validation & Repair
Checked the websites of 43,969 suppliers and repaired the wrong ones with web search and LLMs, cutting the manual review queue from 5,213 to 14.
Data engineer, solo
Jun–Jul 2026
In numbers
- records needing manual review
- 5,213 → 14
- records needing manual review
- of 43,969 suppliers ended with a verified site
- 73%
- of 43,969 suppliers ended with a verified site
- domains repaired per second after a model switch
- 0.1 → 1.2
- domains repaired per second after a model switch
- verified suppliers after the next round
- 574,110
- verified suppliers after the next round
The problem
Some supplier websites on file had shut down, changed hands, or never belonged to the company in the first place. A wrong website means every fact pulled from it is wrong too, and there were thousands of records nobody had time to check by hand.
What I did
- 1
Technical check, then an identity score
An async checker tested DNS, SSL and HTTP for thousands of sites at once; dead ones went straight to repair. Live sites got a 0–100 identity score from their title, description, structured data and copyright line. Above 70 passed, 40–70 was flagged as doubtful, below 40 needed repair.
- 2
Two repair engines in parallel
One engine paired Llama on Bedrock with AWS web search, the other Gemini with Google Search grounding. Both ran side by side, and FOR UPDATE SKIP LOCKED made sure no two workers ever picked up the same record.
- 3
Right-size the model
The repair job started on Llama 3 70B, which kept hitting AWS rate limits, so the pipeline spent most of its time backing off: 0.1 domains a second. All the task needed was to pick the right domain from search results and return JSON. Switching to Llama 3 8B ended the throttling and took it to 1.2 a second, 12× faster. For a 54,000-domain run that meant half a day instead of six, and an estimated cost of about $111 instead of $568.
- 4
Let the LLM settle the doubtful ones
The 5,213 doubtful records were judged by an LLM using the title and description already stored in the database, with no extra web requests. It confirmed 4,791 of them. Dead sites were matched against a 5.3-million-row domain history table I had built earlier, and the remainder were searched again with "automotive" appended to the name.
Result
73% of suppliers ended up with a verified website and the manual queue shrank from 5,213 to 14. 11,843 companies with no trace online were flagged separately. A later round promoted another 10,770 domains, bringing verified suppliers to 574,110.
What I learned
The biggest model is rarely the right one: the smallest model that can do the job is usually faster and cheaper. I'd also calibrate score thresholds like 70 and 40 on a small hand-labelled set rather than picking them by feel.
Stack
- Python asyncio
- PostgreSQL
- Bedrock (Llama 3)
- AgentCore Web Search
- Gemini
- Google Search grounding