The Records Existed

TL;DR

• What happened: asked "How many permits are there?", the evo.ehs AI answered correctly — 75, with examples. Asked the follow-up "Which 10 are due next?", it replied that nothing matched the site's records. The records it had just counted. The same conversation had worked minutes earlier.
• Root cause: the AI had correctly routed the question to SQL — but its generated SELECT was rejected by Postgres, execution fell through to a semantic-similarity gate, and a computed answer has no stored document to score against. The gate structurally converts every SQL failure on an aggregate question into a false "no data."
• The subtle trap: query planning deliberately runs at temperature 0 so routing is deterministic — which means a blind retry reproduces the same broken SQL byte-for-byte. Determinism without a feedback loop just repeats the mistake consistently.
• CAPA: corrective — feed the database's error back to the model for one repair attempt, and replace the misleading refusal with an honest, retryable failure message. Preventive — give the query planner the conversation history, name the ranking-question shape in its prompt, and pin all of it with 12 regression tests.
• Verified: the exact failing conversation replayed five for five, and the full 501-test suite is green. A CAPA isn't closed until its effectiveness is checked — the same rule a plant is held to.

Incident report

• Date of occurrence: August 26, 2026, evening
• System: Ask AI in evo.ehs, answered by the evo-ai RAG service (demo workspace)
• Detected by: the automated documentation-screenshot pipeline — a scripted browser drives a real two-question conversation for the docs, which makes every capture run an end-to-end test
• Classification: near-miss — caught internally before any customer saw it
• Immediate action: the bad screenshot was withheld from publication and an investigation opened

The scripted conversation asks two questions. The first worked perfectly: "How many permits are there?" → "There are 75 permits in total", with examples and links. The second — "Which 10 are due next?", a follow-up that deliberately leans on the previous turn for context — got this:

That doesn't match anything in your site's records. You can ask about
your site's permits, incidents, tasks, chemicals, training, events,
plants & weather, EPA compliance, users, product docs — answers come
only from what's recorded in evo.ehs.

Two things made it worth treating as an incident rather than a shrug. It was a false negative about the user's own data — the worst kind of wrong answer for a compliance product, because "you have nothing due" and "I couldn't check" lead to very different decisions. And it was intermittent: the identical conversation had produced a correct ten-item answer minutes earlier in another session. Intermittent false negatives are the ones that ship.

The evidence

The answering pipeline has layers: a condense step rewrites a follow-up into a standalone question, a planner LLM routes it (SQL for aggregates and rankings, vector retrieval for narrative questions), and anything that falls through to retrieval must clear a relevance gate — a minimum similarity score, there to refuse genuinely off-topic questions without ever invoking the model.

The service logs held exactly one interesting line from the failing run:

analytics query rejected/failed: execution failed:
operator does not exist: date >= text

That one line kills the obvious theory. The planner had not misunderstood the question — it correctly classified "Which 10 are due next?" as a ranking and drafted SQL for it. The SQL was simply wrong: somewhere in the generated SELECT, a date column was compared against a text expression, and Postgres refused it. Here is the code path that turned that rejection into a false "no data" — the error handler logged the failure and then just fell through:

# rag.py — BEFORE
if plan and plan.route in ("sql", "both") and plan.sql:
    try:
        result = await analytics.run_query(tenant_id, plan.sql, source_types)
        analytics_nodes = [analytics.result_node(result)]
        sql_rows = len(result.rows)
    except analytics.AnalyticsError as exc:
        logger.warning("analytics query rejected/failed: %s", exc)
        # ...and that's it. Execution falls through to vector
        # retrieval as if this had never been an SQL question.

nodes = await retriever.aretrieve(expand_query(question))

if below_relevance_threshold(nodes):
    if analytics_nodes:
        nodes = analytics_nodes
    else:
        # off-topic and failed-SQL land in the SAME branch
        return {"answer": GATED_ANSWER, "gated": True, ...}

The fall-through is fatal for this class of question, and it's worth seeing why precisely: "the next 10 due" is computed, not stored. No indexed document contains that sentence — it's a sort over 75 rows that only SQL can produce. So retrieval scores low, the gate trips, and the pipeline reports "nothing matches your records" about records it had counted correctly one question earlier.

Root cause analysis — five whys

1. Why did the user see "nothing matches your records"? The relevance gate refused the question, and the client renders gated answers with its own off-topic wording.
2. Why did the relevance gate judge this question at all? The SQL layer failed and execution fell through to vector retrieval — with no memory that the question had already been classified as an aggregate.
3. Why did no retrieved chunk clear the gate? Structurally, none ever could: a ranking is computed from rows, not stored as a document. The gate was designed to catch off-topic questions; it cannot tell "off-topic" from "the calculation failed."
4. Why did the SQL fail? The model drafted a SELECT comparing a date column with a text-typed expression — operator does not exist: date >= text. LLMs write plausible-but-wrong SQL at some rate; that's a known property, not a surprise.
5. Why wasn't the bad SQL corrected? Planning runs at temperature 0 — deliberately, because routing must be deterministic. But a deterministic model asked the same question writes the same broken SQL byte-for-byte, so a naive retry is worthless, and the pipeline had no path to feed the database's error back as new information.

Root cause: one user-facing message served two unlike failure modes. "This question isn't about your data" and "the calculation over your data failed" collapsed into a single branch, and the message asserted the first while the truth was the second. The latent condition was the shared gate branch; the trigger was one malformed SELECT; and the intermittency came from the condense step — its rewrite of the follow-up varies run to run, so the deterministic planner downstream received slightly different questions and only sometimes produced the fatal SQL.

CAPA — corrective and preventive actions

CA-1 (corrective): repair the SQL with the error as new information

A rejected SELECT now gets exactly one re-plan, with the failed statement and the database's own error in the prompt. This is the key move against temperature-0 determinism: don't retry blind — change the input.

# rag.py — AFTER: one corrective re-plan on rejection
except analytics.AnalyticsError as exc:
    logger.warning("analytics query rejected/failed: %s", exc)
    repaired = await analytics.plan_query(
        llm, question, source_types, allow_actions=allow_actions,
        history=history, failed_sql=plan.sql, db_error=str(exc),
    )
    if repaired and repaired.route in ("sql", "both") and repaired.sql:
        result = await analytics.run_query(tenant_id, repaired.sql, source_types)
        return repaired, [analytics.result_node(result)], len(result.rows)
    # a second rejection gives up on SQL — no retry loops

The prompt block the repair attempt carries:

_REPAIR_BLOCK = """\
A previous attempt routed this question to "sql" with:
  {sql}
The database rejected it: {error}
Write a corrected SELECT that avoids this error — or, if SQL genuinely
cannot answer the question, route it to "vector".
"""

CA-2 (corrective): fail honestly when the calculation still fails

If the SQL stays unusable even after repair, the gate no longer gets to tell the "no data" lie. A question the planner classified as an aggregate now returns an honest, retryable failure instead:

# rag.py — AFTER: the gate can no longer launder a compute
# failure into "no data"
if below_relevance_threshold(nodes):
    if analytics_nodes:
        nodes = analytics_nodes
    elif aggregate_intent:
        # A computed answer has no chunk to clear a similarity
        # gate — "nothing matches" would lie about data that exists.
        return {"answer": ANALYTICS_FAILED_ANSWER, ...}  # honest, retryable
    else:
        return {"answer": GATED_ANSWER, "gated": True, ...}  # truly off-topic

PA-1 (preventive): give the router the conversation

Routing a follow-up used to depend entirely on the condense rewrite keeping the subject — the lossy step that made the failure intermittent. The planner now sees the recent conversation itself (last eight messages, capped), so "which 10" can be resolved against "75 permits" even when the rewrite drops the word:

# rag.py — AFTER: history goes to the planner too
plan = await analytics.plan_query(
    llm, question, source_types, allow_actions=allow_actions,
    history=history,
)

PA-2 (preventive): name the failure shape in the prompt

The planner's routing prompt now states outright that "Which 10 are due next?" and every other next-N-by-date question is a ranking — order by the relevant date column, LIMIT N — even when the record type is only named earlier in the conversation.

PA-3 (preventive): pin it all with tests

Twelve regression tests now cover the seam: that history and repair feedback actually reach the planner prompt, every path through the repair loop (first-try success, repair success, double failure, the model re-routing away from SQL, the repair call itself crashing), and — the one that guards the lesson — that the failure answer never claims the records don't exist.

Effectiveness check

In EHS practice a CAPA isn't closed when the fix is written — it's closed when its effectiveness is verified. Same rule here:

• The exact failing conversation — permits discussed, then the context-only follow-up — replayed against the rebuilt service five times: five correct answers, each the full ten-item due schedule with dates and statuses.
• The service logs show the history-aware planner now writes valid SQL on the first try; the repair loop sits behind it as defense in depth rather than a crutch it leans on.
• Full suite: 501 tests passing, including the twelve new ones.

And the incident's original detector — the documentation-screenshot pipeline — re-ran end to end: both personas now capture the follow-up answered with the complete ranked list.

Why write a bug up like this

Partly because it's the discipline I already live in — evo.ehs exists to make plants do exactly this: report the incident, find the root cause instead of the nearest symptom, and close corrective actions only after checking they worked. Dogfooding the methodology on the product's own AI bug seemed only fair.

But mostly because the format earns its keep on AI systems specifically. The five-whys chain surfaced the two insights a straight bug-fix write-up would have buried: that a similarity gate can never validate a computed answer, so any aggregate pipeline that falls through to one will lie eventually; and that determinism and self-correction pull in opposite directions in LLM systems — temperature 0 bought consistent routing at the price of consistently repeated mistakes, and the only way out is feeding errors back as new input. Those are design lessons, not patch notes.

And one more, straight from safety culture: an error message is an interface, and a wrong one is a hazard. "You have nothing due" and "I couldn't check" are different answers with different consequences — in a compliance product, the gap between them is exactly where trust is lost. The near-miss got caught because a screenshot pipeline doubles as an end-to-end test; the fix made sure that when the system fails again — and it will — it fails telling the truth.

← Back to Blog