Checked search on Telem

Same accuracy. Half the cost.

Backstop asks one engine through Telem, checks the results against your agent's goal, and pays for another engine only when the first one missed.

  • Tavily
  • Exa
  • Brave
  • Perplexity
  • Parallel
  • Ceramic

Live race

One question, two search setups, side by side.

Ask a question and watch two search setups answer it at the same time through Telem. Both sides use the same agent model, prompt and tools; only the search step differs. Every count and cost comes from the run's event log, checker included.

Loading

How it works

Search. Check. Then decide.

Telem's default routing calls several engines before any results exist. Backstop asks one, checks what came back against the goal, and spends another search only on a miss.

Agent asks

search(query, goal)

The agent says what it needs, not only what to type.

Telem searches

The first engine is routed by question type.

  • Perplexity
  • Exa
  • Tavily
  • Brave
  • Parallel
  • Ceramic

One engine at a time, with the goal and the Telem session attached.

Checker

  1. Fast decision modelchecks the results against the goal first.
  2. LLM checkerdecides when the fast model is not confident.
  • Hit
  • Stale
  • Wrong entity
  • Shallow
  • Dead

Rescue on a miss

  • Next enginefrom the measured rescue table, with the query rewritten from the checker's reason
    • Exa
    • then
    • Parallel
  • Read the pagewith Tavily Extract when the right page has a thin snippet (Shallow)

At most two rescues per search, then Not found.

Answer

VerifiedWith the evidence and its citations.
Not foundThe agent declines instead of guessing.

Telem sessions

Every run is grouped in one Telem session, so each search and rescue shows up together in the Telem console.

Tavily Extract

When a snippet is thin, Extract reads the page with the goal as its query and hands evidence chunks to the checker.

Audit trail

Every search, check and rescue is recorded as an event in the run's log, so each answer traces back to what was searched and why.

Results

Measured on held‑out questions.

None of these test questions were used to tune the checker or build the rescue table. Total cost counts search, page reads, the checker and the agent model.

Loading results

Three setups, same test questions

Accuracy bars run from 0 to 100%. The other bars are scaled to the largest value; shorter is better.

Accuracy, total cost, engine calls and median time per question for each setup
SetupAccuracyhigher is betterCost per questiontotal, all modelsEngine callsper questionMedian timeper question

Developers

One line to switch.

BackstopTelem keeps Telem's response shape and adds status, verdict, why, trace, cost and a rank reason on every result. Every other AsyncTelem method passes through.

Python
from telem import AsyncTelem
from backstop import BackstopTelem

client = AsyncTelem()
client = BackstopTelem()

resp = await client.search(
    "What was Stripe's valuation in its 2023 round?",
    goal="find the valuation figure",
)
print(resp.status, resp.results[0].url)
HTTP
curl -s https://backstop-pied.vercel.app/v1/search \
  -H "Authorization: Bearer $BACKSTOP_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "What was Stripe'\''s valuation in its 2023 round?",
       "goal": "find the valuation figure"}'
Install
git clone https://github.com/oliverrowebardeen/backstop && cd backstop
pip install -e .   # Python 3.11+; TELEM_API_KEY, ANTHROPIC_API_KEY and OPENAI_API_KEY in .env