Skip to main content

Critique results and the loop

An agent runs a critique by calling validate with its finished output and one rule or rule set. You get a pass or fail, the passage that broke any failed rule, and what is left to spend. If a rule fails, the agent rewrites that passage and calls again.

The call

validate takes the output and exactly one target.

validate inputs
InputRequiredWhat it does
outputYesThe text to check. Send the whole revised output on every attempt.
rule_id or ruleset_idOne of the twoWhat to check against.
version_idNoWhich version to run. Without it, the active version runs. Your own company can name any version of its own rules, including one not yet active, to test it. Other companies get the active version.
session_idNoNames the loop, so attempts are counted together. Without it, the MCP connection is the session.
max_attemptsNoLowers the attempt cap for this session. It can never raise it.
request_idNoMakes a retry safe. A repeat of a call that already returned a result, with the same output, returns that result and is not charged again.
agent_idNoIdentifies the agent in reports.
tagsNoUp to 10 tags of up to 50 characters, for comparing runs in reports, such as instructions:v2.
agent_modelNoThe model the agent runs on, as it reports it, up to 100 characters. Reports group by it. Vindex does not check it.
agent_labelNoA name for the agent, up to 100 characters.
output_refNoYour own reference for the output, such as a ticket or document ID, up to 200 characters. It is how you match a result to an output later.

Values over these limits are refused before anything is charged, never cut short.

What comes back

A single rule

  • result: pass or fail.
  • On a fail, the passage that broke the rule, quoted. A rule that fails because something is missing has no passage to quote, and the result still names the rule that failed.
  • rule_id and version_id: the exact version that ran.
  • attempt and attempts_left: where the loop stands.
  • credits_left, and low_balance when the balance is low.
  • validation_id: quote it to support if you dispute a result.

A rule set

  • result for the whole set. The set passes only if every rule in it passes.
  • failed_rules: each failed rule with its rule_id, name, and version_id.
  • rules_run: every rule version that was judged, each with its own result.
  • ruleset_id and version_id, plus the same loop and credit fields as above.

A result is a pass or a fail. There is no score. The judgement is probabilistic, so the same output can occasionally get a different result on another run.

The loop

  1. The agent calls validate with its draft.
  2. If a rule failed, it rewrites the quoted passage and calls validate again, with the whole revised output, in the same session.
  3. It stops when every rule passes, when it reaches the attempt cap, or when credits run out, and then tells the person what is still failing.
  • The agent should not resend unchanged output. It is charged like any other attempt.
  • An attempt counts calls in the same session against the same rule or rule set, starting from 1. A call that Vindex released because of an error is not an attempt.
  • The cap applies to each session and target. A call over the cap is refused before anything is held, so it costs nothing, and the agent is told the cap and the rules that failed last time.
  • When the agent stops, the person decides what to do: edit the output by hand, or change the rule.

What a critique costs

  • $0.03 per call, which is one critique credit. A rule set costs the same as a single rule.
  • Each attempt is a separate call and is charged.
  • The credit is held before the engine runs and charged when a result comes back. If Vindex fails after the hold, the hold is released and you are not charged.
  • Refusals happen before anything is held, so they cost nothing.
  • Listing and reading rules, and reading your balance, are free.
  • Credits belong to the company. Every member and agent spends the same balance.

Refusals

Vindex checks the call before it holds any credit and returns the first failure. Each refusal carries a code and names what was wrong. Something you cannot see returns NOT_FOUND, never “forbidden”.

Refusal codes
CodeMeaning
NO_CREDENTIALNot signed in. The result tells the agent how to sign in.
MEMBER_INACTIVEThe member is deactivated.
NOT_FOUNDThe rule or rule set does not exist, or you cannot see it. This includes another company's rule that is in test.
VERSION_NOT_AVAILABLEYou named a version of another company's rule that is not its active version.
RULE_NOT_READYThe rule is a draft. Set it to test first.
RULE_UNAVAILABLEThe rule is disabled or archived, and the refusal says which. For a rule set that holds one, it names the rule.
RULESET_ARCHIVEDThe rule set is archived. The refusal names its replacement, if one is set.
NOT_APPROVEDYour company only allows approved items, and this one is not on the approved list.
OUTPUT_EMPTYThere is no output to check.
OUTPUT_TOO_LARGEThe output is over the size limit.
TAGS_INVALIDMore than 10 tags, or a tag over 50 characters.
FIELD_TOO_LONGA field is over its limit.
ATTEMPT_CAP_REACHEDThe session has used its attempts. Nothing is charged.
TOO_MANY_IN_FLIGHTYour company has too many critiques running at once. Try again shortly.
INSUFFICIENT_CREDITSThe balance cannot cover a critique. The refusal gives the balance, the price, the names of your Company admins, and a link where an admin can buy credits. The MCP server never takes payment.
REQUEST_ID_REUSEDA request_id was reused with different output.

Two further outcomes happen after the hold: ENGINE_FAILED and TIMEOUT. Both say nothing was charged and whether to retry. A call returns within the MCP server’s 70-second limit.

If a result looks wrong

Contact support with the validation_id from the result and the output. Vindex finds the call, the exact rule versions that ran, the engine version, and the charge, and can re-run the output against the same versions. If the engine judged wrongly, support may return the credits.

See also rule sets, reporting, and the first-run guide.