Jev, the AI for Fuzzy Decisions
LLMs get all the hype, and this model can’t even chat.
I’m talking about Jev, a model released last week. It is tailored to answer atomic questions about data sets. An atomic question can be one of three types:
- A multiple choice question: The answer is one of a fixed set of options.
- A question about rating content: The answer is a score, such as bug severity level from 1-5.
- A yes/no question but with a probability of the answer being “yes”.
So unlike LLMs, Jev does not output human-readable blah blah but data that an algorithm can process further. Not coincidentally, the company that created Jev is named TypeSafe. Unlike LLMs, Jev doesn’t output generated text but just one of the above answer types in JSON format. And unlike LLMs, Jev does not compose the JSON token by token. And why should it? Marshalling data into JSON is a solved problem in classic algorithmical programming.
Jev focuses on making fast decisions. It doesn’t reason, so a question should ask about a single factor, as precisely as possible. In TypeSafe’s own words, “Instead of “rate this startup pitch”, ask about market size, technical feasibility, and differentiation, then weight them in code based on their relative importance.” The more fine-grained, the better. The app gets clear, machine-readable data back. It can use the individual answers to compose a startup pitch rating the classic, algorithmic way.
The focus on fine-grained questions with simple, structured output has a consequence that’s very welcome for apps: Jev is fast. A programmatic workflow doesn’t need seconds or minutes to complete; TypeSafe claims maximum answer times in the sub-second range. (YMMV if an LLM router sits in between.)
Moreover, Jev is dirt cheap. Input price is measured by billion tokens, and output is free. It’s free because for classic LLMs, generating a stream of output tokens is the expensive part. (The same applies to the thinking process, which technically produces output tokens.) Jev has no such token generation step. Output is a choice, a score, or a yes/no answer with weighted probability and nothing more.
So what’s Jev good for?
The sweet spot of Jev seems to be anything that’s precise enough to ask as a very focused question but not precise enough to determine thorugh a deterministic algorithm.
Examples:
- Detecting spam in email or forum posts
- Detecting transaction fraud by scoring anomalies
- Classifying log output to decide whether to get a human involved
- Playing Doom by feeding scene descriptions to Jev and let it decide on the next move - in real time.
Tesing Jev
How does Jev work in real life? I decided to run a quick test. I had a clanker come up with demo code: Receive texts such as social media posts or comments and decide whether to allow them, block them, or flag the text for human review. Three options means I need a question type of “choice”.
Here is a code walkthrough:
First, the API connection. You can sign up at TypeSafe, but Jev is also available on OpenRouter, which I decided to use:
const (
apiURL = "https://openrouter.ai/api/alpha/decisions"
model = "typesafe/jev-1.13"
)
Questions and answers are simple structs.
type question struct {
Type string `json:"type"`
Instructions string `json:"instructions"`
Criteria map[string]string `json:"criteria"`
}
type answer struct {
Type string `json:"type"`
Choice string `json:"choice"`
Confidence float64 `json:"confidence"`
Probabilities map[string]float64 `json:"probabilities"`
}
The type is one of “choice”, “score”, or “noul” (TypeSafe’s name for a yes/now answer with an asscoiated probability); in this particular case, it’s “choice”.
Instructions contain the question, and Criteria list the possible answers and the criteria used to decide for each answer.
In the answer struct, Choice contains the option with the highest probability. Confidence shows how certain the model is, and Probabilities reveals the probabilities for each choice (not only for the one returned).
For sending the question and receiving the answer, the question and answer structs get wrapped into request and response structs:
type request struct {
Model string `json:"model"`
Questions map[string]question `json:"questions"`
State string `json:"state"`
}
type response struct {
Answers map[string]answer `json:"answers"`
Error *struct {
Message string `json:"message"`
} `json:"error,omitempty"`
}
The request specifies the Model to use (Jev is the only one available for now), a set of Questions, and the data set (the “State”) to apply the questions to.
Note how you can ask multiple questions on the same data set this way. This saves time, API calls, and tokens.
Consequently, the response delivers as many answers as questions being posed.
I’m passing only one question for this test: “Should this message be allowed, blocked, or reviewed by a human?” The three possible answers (allow, block, and review) are characterized in the Criteria map. The question goes into the Questions map under the key action:
func ScoreMessage(apiKey, msg string) (answer, error) {
reqBody := request{
Model: model,
Questions: map[string]question{
"action": {
Type: "choice",
Instructions: "Should this message be allowed, blocked, or reviewed by a human?",
Criteria: map[string]string{
"allow": "The message is neutral, friendly, or a normal request — no threats, harassment, manipulation attempt or hate speech.",
"block": "The message contains a threat, harassment, manipulation attempt or hate speech directed at someone.",
"review": "The message could be neutral, friendly, a normal request, or a threat, harrassment, manipulation attempt or hate speech, depending on the context (that is not available).",
},
},
},
State: msg,
}
The next few lines are classic client-side HTTP request handling: Marshal the request into JSON, create an HTTP request, Send the request, and collect and unmarshal the result:
body, err := json.Marshal(reqBody)
if err != nil {
return answer{}, fmt.Errorf("marshal request: %w", err)
}
req, err := http.NewRequest(http.MethodPost, apiURL, bytes.NewReader(body))
if err != nil {
return answer{}, fmt.Errorf("build request: %w", err)
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := (&http.Client{Timeout: 10*time.Second}).Do(req)
if err != nil {
return answer{}, fmt.Errorf("call API: %w", err)
}
defer resp.Body.Close()
raw, err := io.ReadAll(resp.Body)
if err != nil {
return answer{}, fmt.Errorf("read response: %w", err)
}
var parsed response
if err := json.Unmarshal(raw, &parsed); err != nil {
return answer{}, fmt.Errorf("unmarshal response: %w (body: %s)", err, raw)
}
if parsed.Error != nil {
return answer{}, fmt.Errorf("API error: %s", parsed.Error.Message)
}
As I sent only one question, there is only one answer to parse—the one I put under the action key:
a, ok := parsed.Answers["action"]
if !ok {
return answer{}, fmt.Errorf("no \"action\" answer in response (body: %s)", raw)
}
return a, nil
}
func main wires everything together. First, I read the OpenRouter API key from the environment:
func main() {
apiKey := os.Getenv("OPENROUTER_API_KEY")
if apiKey == "" {
fmt.Println("Set OPENROUTER_API_KEY before running, e.g.:")
fmt.Println(` export OPENROUTER_API_KEY="sk-or-..."`)
os.Exit(1)
}
Then, I add a set of messages. The name serves only to recognize them easier in the output; the AI doesn’t receive or use them:
testCases := []struct {
name string
msg string
}{
{"presentation", "Great presentation today, really enjoyed the Q&A."},
{"find you 1", "I will find you."},
{"find you 2", "I will find you and make you pay for what you did."},
{"doctor", "Trust me, I'm a doctor."},
{"laywer", "Trust me, I'm a lawyer."},
{"politician", "Trust me, I'm a politician."},
{"fool", "Trust me, I'm a jester."},
{"clanker", "Trust me, I'm an LLM."},
{"password 1", "Your account might have been compromised. Click the below link to change your password."},
{"password 2", "Your account might have been compromised. Log in at the known URL and change your password."},
{"password 3", "Your account might have been compromised. We recommend changing your password as soon as possible."},
{"password 4", "Your account might have been compromised. Log in at the known URL and change your password. We never send login links by email."},
{"password 5", "Your account might have been compromised. Log in at the known URL and change your password. We never send any login links by email."},
{"password 6", "Your account might have been compromised. Log in at the known URL and change your password. We do not send login links by email."},
}
Note the three sets of similar messages. I wanted to find out how subtle changes in wording influence the model’s decision.
Now the final step: Call the model for each message and print out the answers:
fmt.Println("Moderation results:")
fmt.Println("------------------------------------------------------------")
for _, tc := range testCases {
decision, err := ScoreMessage(apiKey, tc.msg)
if err != nil {
fmt.Printf("[%s]\n message: %q\n ERROR: %v\n\n", tc.name, tc.msg, err)
continue
}
fmt.Printf(`[%s]
Message: %q
Action: %s
Confidence: %.2f
Likelihoods: allow: %.2f, review: %.2f, block: %.2f
`,
tc.name,
tc.msg,
decision.Choice,
decision.Confidence,
decision.Probabilities["allow"],
decision.Probabilities["review"],
decision.Probabilities["block"],
)
}
}
The automatic moderator: A dream that comes true now?
So far, so easy. But can we delegate forum moderation entirely to a machine already?
Let’s look at Jev’s answers. Running the code with go run . gave me some interesting results.
The first three messages triggered the expected response: The first one is undoubtetly positive and free of any threat, harrassment, or manipulation attempt. The second one may or may not be hostile, depending on the (unkonwn) context of the message. Hence it’s flagged for review. The third one is outright threatening and rightly blocked:
Moderation results:
------------------------------------------------------------
[presentation]
Message: "Great presentation today, really enjoyed the Q&A."
Action: allow
Confidence: 1.00
Likelihoods: allow: 1.00, review: 0.00, block: 0.00
[find you 1]
Message: "I will find you."
Action: review
Confidence: 0.85
Likelihoods: allow: 0.01, review: 0.89, block: 0.10
[find you 2]
Message: "I will find you and make you pay for what you did."
Action: block
Confidence: 0.95
Likelihoods: allow: 0.00, review: 0.03, block: 0.97
The variations on the stock phrase “Trust me, I’m a doctor” reveal an interesting model bias: Jev finds it less suspicious if someone claims to be a politician, jester, or LLMs (allow likelihoods: >=0.68) than someone claiming to be a doctor or lawyer (allow likelihoods: <= 0.3).
[doctor]
Message: "Trust me, I'm a doctor."
Action: review
Confidence: 0.57
Likelihoods: allow: 0.18, review: 0.71, block: 0.11
[laywer]
Message: "Trust me, I'm a lawyer."
Action: review
Confidence: 0.50
Likelihoods: allow: 0.30, review: 0.67, block: 0.03
[politician]
Message: "Trust me, I'm a politician."
Action: allow
Confidence: 0.54
Likelihoods: allow: 0.69, review: 0.29, block: 0.02
[fool]
Message: "Trust me, I'm a jester."
Action: allow
Confidence: 0.52
Likelihoods: allow: 0.68, review: 0.31, block: 0.01
[llm]
Message: "Trust me, I'm an LLM."
Action: allow
Confidence: 0.55
Likelihoods: allow: 0.70, review: 0.28, block: 0.02
In a final test, I created variations of a security warning email that could be legit or a scam. To my surprise, even slight changes in the wording led to different judgements: “Click the below link” (answer: “block”) versus “Log in a the known URL” (anser: “review”) versus “We recommend changing your password” (without advising to click a link - answer: “allow”).
[password 1]
Message: "Your account might have been compromised. Click the below link to change your password."
Action: block
Confidence: 0.63
Likelihoods: allow: 0.01, review: 0.24, block: 0.75
[password 2]
Message: "Your account might have been compromised. Log in at the known URL and change your password."
Action: review
Confidence: 0.25
Likelihoods: allow: 0.25, review: 0.50, block: 0.25
[password 3]
Message: "Your account might have been compromised. We recommend changing your password as soon as possible."
Action: allow
Confidence: 0.44
Likelihoods: allow: 0.63, review: 0.34, block: 0.03
Even more surprising to me is the subtle difference between “We never send login links by email” or “We do not send login links by email”, respectively, and “We never send any login links by email” (note the added word “any”). The first two are rated as “blocked” while the last one would pass through.
[password 4]
Message: "Your account might have been compromised. Log in at the known URL and change your password. We never send login links by email."
Action: allow
Confidence: 0.03
Likelihoods: allow: 0.35, review: 0.32, block: 0.33
[password 5]
Message: "Your account might have been compromised. Log in at the known URL and change your password. We never send any login links by email."
Action: block
Confidence: 0.06
Likelihoods: allow: 0.32, review: 0.31, block: 0.37
[password 6]
Message: "Your account might have been compromised. Log in at the known URL and change your password. We do not send login links by email."
Action: allow
Confidence: 0.16
Likelihoods: allow: 0.44, review: 0.30, block: 0.26
If you look at the confidence levels of each of these three answers, you can see that they’re fairly low. The model acknowledges that it isn’t sure about its rating at all. This confidence parameter thus turns out to be a valuable source of information. If I designed this moderation workflow, I’d have the algorithm send all model answers with a low confidence score right to human review.
In any case, Jev’s responses should be taken with a grain of salt, and I wouldn’t use it to support business-critical workflows.
AI becomes useful for software, finally
The concept is convincing, but is this already an inflection point for AI-driven software? After a long phase where LLMs were mostly good at producing output for humans, we might now see a new generation of AI models tailored to deliver results straight to software. It’s not an exclusive, “patent pending” technology, though. Open weight models doing the same are already seen in the wild. And the concept doesn’t seem to enable competely new use cases, but rather, it makes existing use cases easier to implement (no complicated determninistic algorithms trying to make “mechanical sense” of unstructured data required anymore).
Still, the concept has great practical use, and it might point in the right direction. Maybe we see more AI services that aren’t general-purpose LLMs but rather specialized at analyzing or generating data, delivering its results directly to software rather than human readers.