Intent Classification: The Front Door of a Chatbot

#The scenario

A chatbot receives everything from "what are your hours" to multi-paragraph technical complaints. Today most teams send all of it to a frontier LLM with a giant system prompt that tries to both route and answer. The result: premium token prices for messages a lookup table could have handled, and routing logic buried in a prompt nobody can test.

#Why Jev fits

Intent classification is the canonical closed-world problem: you can enumerate the intents in advance (faq / order_status / troubleshooting / cancel / human_agent / …), every message gets exactly one, and the decision must be fast because the user is staring at a typing indicator. Choice returns per-intent probabilities plus confidence — low confidence can mean "this intent taxonomy needs a new branch," a signal prompt-based routing never surfaces cleanly.

#Judgment design example

Structured input per message:

Judgments to define:

1. Choice: "What is the user trying to do?" → one of your enumerated intents. Confidence below threshold → a clarifying-question handler, not a guess.
2. Noul: "Does this message require a human agent?" → yes → straight to the handoff flow, skipping automation entirely.
3. Score: "Frustration level 1–5" → high frustration bumps queue priority even when the intent is routine.

#Where the LLM sits

Behind the router, and only for the intents that need it. order_status hits an API and a template; faq hits a retrieval answer; only genuinely open-ended messages (troubleshooting, unclassifiable) reach the LLM with full context. The LLM's system prompt shrinks from "router + every answer mode" to "answer this specific kind of question" — cheaper per call and easier to evaluate. See the cascade routing recipe.

#Watch-outs

• Multi-intent messages. "I want to cancel and get a refund" violates one-of-N. Either model the dominant intent explicitly in the instructions, or add combination intents for your common pairs.
• Keep the taxonomy honest. Choice supports up to 255 options, but accuracy on 200 barely-distinguishable intents will disappoint you; group aggressively and split later with a second-stage classifier.
• Measure per-intent accuracy. Early independent testing found accuracy varies widely by task — an intent set that works in a demo may fail on your actual message distribution. Log verdicts, audit weekly.

#Sources

• learnjev.com — Three Primitives tutorial (Choice: ≤255 options, probabilities + confidence) and cost/benchmarks context for per-message economics.
• jevai.wiki — model and API pages (latency 70–500 ms figures, no SLA — relevant for typing-indicator budgets).
• "Jev is HERE. How to use it" — Greg Isenberg (~293K views) — the "built for classification" framing.
• AI with Surya — early-access coverage: judgment, not generation, as the product.

Unofficial fan-made handbook. Not affiliated with TypeSafe AI or jev.com.