What you will get
Go through your AI agent's run logs and you will see that most of its time is not spent writing. It is making small decisions: which tool to use next, whether a piece of material is useful, whether a task is done. Each small decision is billed at the price of the most expensive large model. This tutorial shows you how to hand these decisions to Jev, a model that only answers multiple-choice questions, and to pass the ones it is unsure about to a large model. The article includes 3 prompts you can copy and give to your agent as-is.
The large model stays. It just does less of the grunt work. Writing text and code is still the large model's job. Judgments that pick one option from a list go to Jev, which is cheap and fast.
What Jev is
Jev is a model from TypeSafe AI. According to the official description, it does not write text. It only answers questions where you list the options in advance, and every answer comes with a probability. For example, if you ask "Is this email a scam?", it does not write an analysis. It returns a single probability for "yes", such as 0.97.
The probability is useful because Jev is trained so that when it says it is 90% confident, the answer is right about 90% of the time. Your program can use this: act directly when confidence is high, and send it to someone else to review when confidence is low.
It currently supports several question types:
| Type | What it asks | What it returns |
|---|---|---|
| Choice | Pick one from the options you list | The option picked, and a probability for each option |
| Score | Score against the criteria you write in a few levels | A score (it can fall between two levels), and a probability for each level |
| Noul | Yes or no | The probability of "yes" |
| Box | Where something is in an image | The position of a box (only in newer versions) |
Per the official model page, the current jev-1.14 and jev-1.13 are priced the same: $0.042 for every million tokens you send in. One request can hold about 64K tokens. The official site says that in its own tests it is about 194 times faster and about 445 times cheaper than general-purpose large models. These are the vendor's own numbers. Measure the real results yourself.
Step 1: have the agent sort the workflow into four categories
Don't rush to connect the API. Find a run log from your agent that really ran, and give it to Claude Code or Codex:
Read this agent run log: [file path]
Sort each step in it into one of the four categories below. Make a table (step, category, reason, roughly how many times this step happens per day):
1. Steps that write text, code or plans: keep them with the large model
2. Steps with fixed rules (counting, date math, retry counts, allowlists): turn them into code
3. Steps that pick one from a set of options you list in advance (classification, keep or drop, safe or unsafe, done or not done): candidates for Jev
4. Steps that cannot be undone (sending, paying, publishing, deleting, changing permissions): must be confirmed by a person
Finally, sort category 3 by how often it happens, from most to least, and list the top 5. Analyze only. Do not change any code.
Category 3 is usually bigger than you think. It is the part this change targets. Turning category 2 into code along the way also saves you the Jev cost.
Step 2: start with one decision
Pick one decision from category 3 that comes up dozens of times a day and is not a disaster when it's wrong, such as "which department should this email go to". Don't start with something like "should we issue a refund", where a mistake costs money.
Step 3: write the decision as a good question
How well Jev answers depends mostly on how well the question is written. Have the agent write the question following the rules below, and connect it to the API:
Sign in to unlock this key section
New accounts get 10 free credits, plus 1 more for each daily check-in. Spend 1 credit to unlock and it stays open for you.
Rules 1 to 5 come from Jev's advice on writing questions. The most important part of the whole approach is rule 8: when Jev is unsure, don't let it guess.
Step 4: route by confidence
Once it is connected, every decision goes through the four layers below in order, and stops at the first layer where it is confident:
The threshold is set by the action, not by the model. Filing a newsletter can be done automatically at 0.7 confidence. Flagging a scam may need 0.95 or higher. For things that can't be undone, such as deleting or paying, leave them to you no matter how confident the system is.
Step 5: do a one-week shadow run first
Don't switch the next day. First, do a one-week shadow run in which Jev only listens in: it gives its answers and probabilities as usual, but decisions are still made the old way. After a week, have the agent compare:
Read this week's logs. Split them into bands by Jev's confidence (0.9 and above, 0.7 to 0.9, below 0.7).
For each band, count how often Jev's answer matches the original decision, and estimate how much money this week would have saved if Jev had decided first.
Do not change any config. Give me a report only.
For any band where the match rate is close to 100%, switch that band over first.
Work out the cost
At official prices, 10,000 decisions with 1,000 input tokens each cost Jev about $0.42. Suppose 10% of them are uncertain and go to the large model. Then the total cost is mostly those 10% of large-model calls. How much you save depends on which model you used for each decision before. The shadow-run week in step 5 gives you the real numbers.
Notes
- It is still new: TypeSafe only went public in September this year. The product and prices change quickly, so follow the official docs.
- Speed and price multiples are the vendor's own tests: don't use them directly for a budget. Use your own shadow-run data.
- An answer can be inside the range and still be wrong: Jev never gives an answer outside your options, but it can pick the wrong option. The thresholds and the large-model fallback exist for this.
- Data goes to a third party: for sensitive information such as user emails and orders, first confirm that your data is allowed to be processed by them.
Comments (0)
No comments yet. Start the conversation.