AI / ML · 2026
Honest
A production LLM feature is only trustworthy if something grades every answer before a user sees it. Honest is that layer — a working evaluation harness that catches hallucinations, ungrounded claims, and broken guardrails, and fails the build when a prompt change regresses.
Why this exists
Shipping AI features into Parkzy taught me the real failure mode isn't the model being dumb — it's the model being confidently wrong. A pricing explanation that invents a fee. An availability answer that hallucinates a time slot. A support bot that follows a prompt injection. The model sounds fine; the answer ships a bug.
My fix in production was to ground the LLM in code-computed facts — the deterministic engine calculates every number, the model only explains what's already true. Honest is that instinct turned into a reusable gate: nothing reaches a user unless it passes the checks.
Grade a Model, Live
Pick a production feature, then pick a candidate answer. Honest runs its checks in your browser and flags anything ungrounded — try the bad answers and watch it catch the hallucination before a user would.
The checks are real functions running client-side on a seeded suite — nothing is hardcoded per answer. Hover a flag to see why it failed.
The checks
Groundedness
Every factual claim — numbers, fees, amenities, times — must trace to the provided facts. Anything the model invented gets flagged and highlighted. This is where most hallucinations die.
Correctness
Beyond grounded: is the answer actually right? A per-case assertion checks the true thing was said (the price breakdown adds up; the availability answer says “no”).
Guardrail / safety
Does it refuse out-of-scope and prompt-injection requests instead of complying or leaking the system prompt? Detects both the leak and the obey.
Regression gate
The whole suite runs as a CI-style gate — a single red bar means a prompt or model change regressed a case. Fail the build, not the user.
How it slots in
Honest runs on every prompt or model change, in CI, before deploy. Deterministic checks (grounding, format, safety patterns) run instantly and for free; subjective dimensions can add an LLM-as-judge pass. A failing check blocks the merge, and every answer gets a scored trace — the observability layer for an AI feature. It's model-agnostic: point it at any provider's output.
Honest about Honest
This is a working demonstration harness — the three checks above run live, client-side, on a seeded suite, and nothing is hardcoded per answer. It is not a battle-tested platform at production traffic. It's the eval layer I'd build on a production AI team, here as a runnable proof of the approach — because the fastest way to show you can keep AI honest is to hand you the thing that does it.