Guides

Probabilities & confidence

Every answer is a probability distribution read directly from the model, not a sampled string. That makes thresholds meaningful.

Probabilities

For choice and score questions, probabilities covers every option and sums to 1. For noul, the single number is the probability of yes. The choice field is simply the most likely option.

Confidence

confidence summarizes how peaked a distribution is, using its entropy H over K options:

Formula

confidence = 1 − H(p) / ln K

It is 1 when all probability is on one option and 0 when the distribution is uniform. For example, [0.84, 0.159, 0.001] has a confidence of about 0.60: the top option is likely, but a real alternative remains.

Uncertain answers

When any answer in a read is uncertain, OpenJev reads it again with fresh noise, up to four times, and averages the distributions. This smooths out noise without changing confident answers. Only the first read is billed.

Using thresholds

Python

answer = response.choices["department"]
if answer.confidence >= 0.8:
    route(answer.choice)
else:
    send_to_human_review()

if response.nouls["is_urgent"].noul > 0.7:
    page_on_call()

Pick thresholds on a labelled sample: raise them for precision and lower them for recall.