HF Agentic Search: Evidence-Led Dataset Discovery
- parse brief
- plan searches
- read cards + rows
- evidence weak?
- re-search
Finding a dataset on the Hub is easy; deciding whether it can actually support the model you want to train is not, and popularity is a bad proxy for it. This turns a plain-language project brief into a small research loop: parse the brief into language, modality, task, schema, license and size constraints, plan several targeted Hub searches, then open the candidates and read them — dataset cards, tags, configs, splits, schema fields and sample rows from the Dataset Viewer. When the first pass returns weak evidence it reflects and runs a second, narrower search rather than returning the best of a bad set. The output is a ranked shortlist where every position has a stated reason attached to something in the card.
- Loop
- Plan → inspect → reflect → re-search
- Ranks on
- Schema evidence, not downloads
Searching the Hugging Face Hub is a solved problem. Deciding whether the dataset you found can actually support the model you want to train is not, and the two get confused constantly.
Hub search ranks by text match and popularity. Neither tells you whether the splits are usable, whether the license permits what you intend, whether the schema has the field your loss function needs, or whether the rows look anything like the examples in your head. Those answers are all sitting in the dataset card and the Dataset Viewer — they just require someone to open them.
The loop
This is a small research agent, not a search box with an LLM in front of it. Given a plain-language brief, it:
- Parses the brief into constraints — language, modality, task, schema, license, size, intended use. These become filters with reasons attached, not keywords.
- Plans several targeted searches rather than one. A brief usually implies three or four different phrasings that surface disjoint candidate sets.
- Deduplicates and pre-ranks by relevance to the parsed request, not by downloads.
- Opens the candidates. Cards, tags, configurations, splits, schema fields, sample rows.
- Reflects on weak evidence. If the first pass comes back thin, it runs a second, narrower search instead of returning the best of a bad set.
- Returns a ranked shortlist where every position carries a stated reason tied to something in the card.
Step 5 is the one that makes it agentic in a way that matters. A single-pass pipeline always returns something; its worst output looks exactly like its best output. Letting the loop notice that the evidence is weak and go again is what separates a shortlist from a guess.
Ranking on evidence
| Signal | Where it comes from | What it rules out |
|---|---|---|
| Schema fields | Dataset Viewer / configs | Datasets missing the target column |
| Split structure | Configs | No held-out set to evaluate against |
| License | Card metadata | Anything incompatible with the stated use |
| Sample rows | Dataset Viewer | Cards that describe data the rows don't contain |
| Size | Card metadata | Too small to train, too large to iterate |
The last row of that table is the one that surprises people. A dataset card is a claim, and the sample rows are the evidence for it. They disagree often enough that reading the rows is not optional — which is precisely the kind of check that is tedious for a human and cheap for an agent.
Where it runs
The source is on GitHub and the app runs as a Gradio Space. It was built for the Hugging Face build-small hackathon, and it uses SmolLM2-360M-Instruct for the parsing and reflection steps — the constraint of the format, and a reasonable one: the hard part here is the loop and the evidence, not the size of the model reading a dataset card.
The general shape
This is the same argument as everything else on this site, applied to a different surface. Retrieval that ranks by popularity is retrieval that cannot tell you when it has nothing good. Adding a reflection step and a stated reason per result costs one extra round trip and converts a ranked list into something a person can audit.