Recipe: Speculative Fan-Out — Many Questions, One Call

The pattern: instead of calling Jev once per question, pack every small question you have about the same item into one request's questions map. One HTTP round trip, one state transmission, N typed verdicts back — evaluated as a batch.

#Why it works

Two cost structures line up in your favor:

1. You pay input tokens once. Input is the only metered dimension ($0.042/M tokens; output is free). Five separate calls each re-send the same state — fan-out sends it once.
2. Latency stays flat. One 70–500 ms call instead of five sequential ones (or the operational complexity of five parallel ones). At 1200 requests/min, you also spend 1 request instead of 5 against the rate limit.

"Speculative" because you typically ask more questions than you'll strictly need — at these prices, asking "is it spam?" and "which folder?" and "how urgent?" in one shot beats the latency of asking the second question only if the first answer requires it.

#The recipe

{
  "model": "jev-1.13.0",
  "state": {
    "from": "no-reply@secure-verify-example.com",
    "subject": "URGENT: your account has been suspended",
    "body_excerpt": "Dear customer, verify your password within 24 hours..."
  },
  "questions": {
    "is_phishing": {
      "type": "noul",
      "instructions": "Answer yes if this email shows signs of phishing: urgency pressure, credential requests, mismatched links."
    },
    "intent": {
      "type": "choice",
      "instructions": "Pick the sender's primary intent.",
      "options": ["credential_theft", "marketing", "notification", "support_reply", "other"]
    },
    "urgency_for_user": {
      "type": "score",
      "instructions": "Rate how time-sensitive this email is for the recipient.",
      "scale": 5
    }
  }
}

One call returns all three verdicts. Your code then composes them: is_phishing above threshold → quarantine regardless of the rest; otherwise intent routes and urgency_for_user sorts.

#When to fan out — and when not to

Fan out when:

• The questions all concern the same item (same state). That's the whole trick — shared state, many judgments.
• The questions are cheap to answer speculatively relative to the latency of asking sequentially.
• Downstream logic might need any subset of the answers; compute now, branch later.

Don't fan out when:

• Later questions depend on earlier answers ("if it's phishing, ask which brand it impersonates"). That's a cascade — see Cascade Routing — and trying to flatten it into one call produces garbage answers to questions whose premise didn't hold.
• The questions need different states. Different state = different call, by definition.
• You're near the limits: state + longest question must fit in 32k tokens, and enormous question maps per request are asking for timeouts. Batch sensibly (dozens of small questions, not thousands).

#Combining with gating

Fan-out pairs naturally with Confidence Gating: ask three questions, act only where every relevant verdict clears its threshold, and send the whole bundle (all three answers, all probabilities) to the fallback reviewer when any one of them is uncertain. The reviewer sees a richer case file than any single question would have produced.

#Where to go next

• The Three Primitives — mixing noul/choice/score in one map
• Confidence Gating — what to do with the batch of verdicts
• API Reference — the questions map contract

#Sources

• learnjev.com — Three primitives tutorial (community documentation; mixing primitives per request).
• jevai.wiki — API reference (community documentation; request structure and limits).
• jev101.com — 什么是 Jev（中文） (community documentation, Chinese; application patterns).

Unofficial fan-made handbook. Not affiliated with TypeSafe AI or jev.com.