Ferhat RudvanoğullarıData Engineer
← All work

Data quality & master data

Domain Validation & Repair

Checked the websites of 43,969 suppliers and repaired the wrong ones with web search and LLMs, cutting the manual review queue from 5,213 to 14.

Data engineer, solo

Jun–Jul 2026

In numbers

records needing manual review
5,213 → 14
records needing manual review
of 43,969 suppliers ended with a verified site
73%
of 43,969 suppliers ended with a verified site
domains repaired per second after a model switch
0.1 → 1.2
domains repaired per second after a model switch
verified suppliers after the next round
574,110
verified suppliers after the next round

The problem

Some supplier websites on file had shut down, changed hands, or never belonged to the company in the first place. A wrong website means every fact pulled from it is wrong too, and there were thousands of records nobody had time to check by hand.

What I did

  1. 1

    Technical check, then an identity score

    An async checker tested DNS, SSL and HTTP for thousands of sites at once; dead ones went straight to repair. Live sites got a 0–100 identity score from their title, description, structured data and copyright line. Above 70 passed, 40–70 was flagged as doubtful, below 40 needed repair.

  2. 2

    Two repair engines in parallel

    One engine paired Llama on Bedrock with AWS web search, the other Gemini with Google Search grounding. Both ran side by side, and FOR UPDATE SKIP LOCKED made sure no two workers ever picked up the same record.

  3. 3

    Right-size the model

    The repair job started on Llama 3 70B, which kept hitting AWS rate limits, so the pipeline spent most of its time backing off: 0.1 domains a second. All the task needed was to pick the right domain from search results and return JSON. Switching to Llama 3 8B ended the throttling and took it to 1.2 a second, 12× faster. For a 54,000-domain run that meant half a day instead of six, and an estimated cost of about $111 instead of $568.

  4. 4

    Let the LLM settle the doubtful ones

    The 5,213 doubtful records were judged by an LLM using the title and description already stored in the database, with no extra web requests. It confirmed 4,791 of them. Dead sites were matched against a 5.3-million-row domain history table I had built earlier, and the remainder were searched again with "automotive" appended to the name.

Result

73% of suppliers ended up with a verified website and the manual queue shrank from 5,213 to 14. 11,843 companies with no trace online were flagged separately. A later round promoted another 10,770 domains, bringing verified suppliers to 574,110.

What I learned

The biggest model is rarely the right one: the smallest model that can do the job is usually faster and cheaper. I'd also calibrate score thresholds like 70 and 40 on a small hand-labelled set rather than picking them by feel.

Stack

  • Python asyncio
  • PostgreSQL
  • Bedrock (Llama 3)
  • AgentCore Web Search
  • Gemini
  • Google Search grounding