Read options

Overview

These fields are OpenJev additions and come from the example server in vLLM PR #57250. The TypeSafe SDKs never send them, so SDK code is unaffected.

steps

A read normally takes one denoise step, so every answer is read at once. With steps above 1 the model denoises the answer slots several times, and each answer can settle against the others. Try it when questions are related, for example a category followed by a subcategory. The token cost is the same; each extra step adds GPU time.

samples

What happens if you leave it out

The server reads your request once. It then measures the uncertainty of each answer. If any answer is uncertain, the server reads three more times with different noise and averages the four results. The policy is all or nothing: one read, or four reads. One uncertain question causes a re-read of every question in the request.

These extra reads are the server's own policy. They do not count against your quota, and usage.input_tokens is the same either way. They do take time. On one RTX PRO 6000, a request with 3 questions takes about 31 ms with one read and about 81 ms with four reads. The four reads run together, so the request takes about 2.6 times one read, not four times.

The extra reads make the answers more accurate. We tested 41 drawn images with exact answers, 3 repeats each. The four-read default scored 88.7%. One read scored 85.1% and 85.7% in two runs. The mean confidence was the same. All of the gain was in questions that count objects, which is where a single read is least stable. 14 of the 17 question families scored the same with one read and with four.

Set it yourself

samples: 1 pins one read. The answer arrives in about half the time, at the same token cost. Use it when your questions are clear and latency matters.

samples: 8 reads the same request eight times with different noise. It averages the eight distributions before it picks choice and score and computes confidence. It gives smoother probabilities, which helps when you set thresholds. You pay for each read, so 8 samples cost 8 times the input tokens.

think

think lets the model reason in text before it answers. OpenJev generates a thought of at most think tokens, then reads the answers with the thought in context. It helps with arithmetic and multi-step problems:

{
  "model": "openjev-latest",
  "state": "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.",
  "think": 512,
  "questions": {"ball_5c": {"type": "noul", "instructions": "The ball costs 5 cents."}}
}

The budget is a hard cap: a thought that reaches it is cut off mid-sentence. Give multi-step problems 512 or more. The input is read twice (once to think, once to answer), and the thought tokens are reported as usage.output_tokens. Both count toward your System One quota.

sequential

Large question sets are answered in parallel chunks, and a chunk can't see the others' answers. sequential: true reads the chunks in order instead, and each chunk sees the answers already chosen. Use it when later questions depend on earlier ones, for example tagging every word of a sentence. It is slower, because the chunks can't run in parallel.

Images

steps and samples work with images. think and sequential need a text-only state; combining them with images returns 422.

Try it

Every option is in the Advanced panel of the Raw playground, with a preset for each.