How do I test a rule before my agents rely on it?

A new rule starts as a draft, and your agents cannot run a draft. That gives you room to check it before anyone relies on it. The test status is the step in between: your company can run the rule by calling it directly, but it does not appear in the list your agents read.

1. Set the rule to test

In Agent Rules, change the status, or ask your agent to do it with update_rule. Until you do, a critique against the rule is refused with RULE_NOT_READY, and nothing is charged. No other company can reach your test rule. They get “not found”.

2. Run an output that should pass

Ask your agent to check a draft you know meets the rule, naming the rule by its ID. You are checking that the rule does not flag good work.

3. Run an output that should fail

Then check a draft you know breaks it. Read the quoted passage: is it the words you would have pointed at? If a borderline case matters to you, run that too.

Each run is a critique and uses a credit, so keep the set of test outputs short and reuse it. Test runs are marked as tests, so reports leave them out unless you ask for them.

4. Revise, or set it to active

If a result is wrong, the rule is usually ambiguous. Upload a revised version, which saves as a new version, and check again. When the rule behaves as you intend, set it to active. From then on it is in the list your agents read.

Keep your pass and fail outputs. The next time you change the rule, run them again before you set the new version to active, so a change in wording does not quietly change what the rule catches. To change a rule without disturbing agents, see rule format: agents keep getting the active version until you set the new one.