How a decision is made
Triggers set the verdict, severity ranks the content, your content rules decide first and triage decides the rest. reasons records the path taken.
A moderation answers two questions. flagged says whether a policy you asked to flag has fired: the SAFE or FLAGGED verdict. recommendation says what to do with the content: allow, review or reject. Both are computed in a fixed order from the scores, the template and the author, and reasons records every step that mattered.
They are separate answers. Content can be FLAGGED and allowed, because a rule lets trusted authors through. Content can be SAFE and sent to review, because its score sits between two thresholds.
{
"flagged": true,
"recommendation": "review",
"severity": "critical",
"policies": {
"personal_data": 0.75,
"links_contact": 0.9,
"instructions": 0.8,
"guideline/no-off-site-deals": 1 // sixteen more policies, all at 0, are trimmed
},
"triggered": ["personal_data", "guideline/no-off-site-deals", "instructions"],
"rule": { "id": "r6", "name": "Links from new accounts", "action": "review" },
"reasons": [
"trigger:personal_data:flag",
"trigger:guideline/no-off-site-deals:flag",
"trigger:instructions:flag",
"rule:r6"
]
}This is the listing description from Moderate text: a link to another shop and a phone number, sent by an author Infrared had not seen before. Its severity is the highest there is, and still the recommendation is review, not reject. The steps below show why.
The scores the decision reads: personal_data0.75 matchlinks_contact0.90 matchinstructions0.80guideline/no-off-site-deals1.00
The author: user_42status enabledtrust level new
Triggers
Each policy that can trigger, against its threshold.
Flagged- Fired:
personal_dataa match - Fired:
guideline/no-off-site-deals1.00 at 0.70 - Fired:
instructions0.80 at 0.70 - Did not fire:
links_contact0.90, its trigger is ontitle
- Fired:
Severity
The highest score times its weight.
1.00 × 1, socriticalContent rules
Top to bottom. The first match decides.
r1 to r5 do not matchr6decidesReviewSeverity triage
Only when no rule matched.
Not asked:Skipped, would be rejectclient_actionAn escalation or an override sent with the request.
Nothing to apply:None sentPartial floor
Only when a policy did not answer.
Nothing to lift:Every policy answered
flagged: truerecommendation: "review"reasonsin the order they were written
trigger:personal_data:flagtrigger:guideline/no-off-site-deals:flagtrigger:instructions:flagrule:r6
Sample data
The order
- Validation. A submission that fails it is rejected here and nothing below runs.
- Triggers decide
flagged. - Severity is computed from the scores.
- Content rules are checked top to bottom. The first one that matches sets the recommendation.
- Severity triage sets the recommendation when no rule matched.
client_actionfrom the request can change the recommendation.- The partial floor applies when a policy did not answer.
Triggers decide flagged
A trigger watches one policy and has an effect, flag or review. It fires when the policy's score reaches its threshold, which is 0.70 for most policies, and on any match for the pattern policies such as personal_data and links_contact.
- When a trigger with the flag effect fires,
flaggedistrue. A trigger with the review effect leavesflaggedalone and adds a review signal, used by triage below. - Every policy in trigger mode has a trigger. Where the template sets none, the policy gets one for the whole template. A policy in observe mode is scored and never fires.
- A trigger set on a field replaces the one the policy would get by default. Unless the template also sets a trigger for the whole template, the policy then fires on that field and nowhere else.
- A single object sent without
fieldis held to the triggers of every field.
The sample shows the third point. links_contact scored 0.9, but the template triggers it on title only and the link was in description, so it is missing from triggered. The three policies that are listed had no trigger of their own and fired through the default one. Triggers has the rule in full.
Severity ranks the content
Severity is the highest score after each is multiplied by its policy's weight. Safety policies such as toxicity count in full, links_contact at 0.95, spam at 0.8, and quality signals such as image_quality at 0.5, unless the template sets other weights. A match on a pattern policy counts as at least 0.85 before its weight, so one link or one phone number ranks high. Observed policies do not count.
In the sample the guideline scored 1 at full weight, which makes the severity critical. The response carries the band, not the number:
severity | The weighted score is |
|---|---|
none | exactly 0 |
low | below the review threshold |
medium | at or above the review threshold, 0.40 by default |
high | at or above the reject threshold, 0.80 by default |
critical | 0.95 or more |
Rules first, triage for the rest
Your content rules read the scores, the author's status and trust level, the values of fields and the language. They are checked top to bottom, and the first rule whose conditions all hold sets the recommendation. rule in the response names it.
Inside a rule, "is flagged" means the score reached the policy's own threshold. It does not look at triggers, so a policy you only observe can still drive a rule. That is how the sample's rule, "Links from new accounts", saw links_contact although no trigger fired for it.
When no rule matches, severity triage decides: allow below the review threshold, review from it, reject from the reject threshold. If triage says allow and a trigger with the review effect fired, the recommendation becomes review. The basic check has no rules, so triage decides everything there.
Triage alone would have rejected the sample. The rule matched first, so it goes to a reviewer instead.
A request without author_id is judged as an author with status enabled and the lowest trust level, new.
Change the result from your side
Your backend sometimes knows something Infrared does not: the account was reported an hour ago, or a staff member is posting. Send client_action with the request to act on that.
client_action.actionstringRequiredescalateapplies your recommendation only when it is stricter than the computed one.overridereplaces the computed one.One of
escalateoverrideclient_action.recommendationstringRequiredThe recommendation you want.
One of
allowreviewreject
curl https://api.useinfrared.com/v1/moderations \
-H "Authorization: Bearer $INFRARED_KEY" \
-H "Content-Type: application/json" \
-d '{
"type": "text",
"input": "Do you deliver to the city centre?",
"author_id": "user_42",
"client_action": {
"action": "escalate",
"recommendation": "review"
}
}'const response = await fetch("https://api.useinfrared.com/v1/moderations", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.INFRARED_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
type: "text",
input: "Do you deliver to the city centre?",
author_id: "user_42",
client_action: {
action: "escalate",
recommendation: "review",
},
}),
});
const moderation = await response.json();import os
import requests
response = requests.post(
"https://api.useinfrared.com/v1/moderations",
headers={"Authorization": f"Bearer {os.environ['INFRARED_KEY']}"},
json={
"type": "text",
"input": "Do you deliver to the city centre?",
"author_id": "user_42",
"client_action": {
"action": "escalate",
"recommendation": "review",
},
},
)
moderation = response.json(){
"object": "moderation",
"id": "mod_4vpwwLeQiXC0EA2C2EUdF7",
"status": "completed",
"flagged": false,
"recommendation": "review",
"severity": "none",
"triggered": [],
"rule": null,
"reasons": ["triage:allow", "client_action:escalate:review"]
}Every policy scored this message 0 and triage allowed it. The escalation then raised the recommendation to review. Only the recommendation changes: flagged, triggered and rule stay as computed, and a review result enters the review queue as usual.
When a policy did not answer
A moderation whose status is partial was decided without one or more policies. If one of them was in trigger mode and the recommendation would be allow, it becomes review, so nothing passes unscreened. A recommendation set by a content rule is left as it is.
This step runs last. When no rule decided, it also turns a client_action that overrides to allow back into review.
Every value of reasons
reasons lists the steps in the order they happened.
| Value | Meaning |
|---|---|
validation_failed | Validation failed. It is the only reason on an invalid moderation. |
trigger:toxicity:flag | A template-wide trigger fired: the policy, then its effect, flag or review. |
trigger:toxicity@description:flag | A trigger set on a field fired: the policy, the field, the effect. |
rule:r6 | The content rule with this id decided. |
triage:allow, triage:review, triage:reject | No rule matched and severity triage decided. |
review_trigger | A trigger with the review effect turned triage's allow into review. |
client_action:escalate:review | The escalation in the request took effect. The last part is the new recommendation. |
client_action:override:allow | The override in the request replaced the recommendation. |
partial:toxicity,spam | These policies did not answer. One entry, keys separated by commas. |
partial_review | A policy that did not answer turned allow into review. |
An escalation that is not stricter than the computed result changes nothing and adds no reason.
Next
- Content rulesOrdered rules on a template turn scores, author trust and field values into allow, review or reject. Severity triage decides when no rule matches.
- TriggersA trigger says what a policy's score sets off, a flag or a review, for the whole template or for one field. One of your own replaces the default.
- SimulationReplay the last seven days of a template's decisions against its draft and count what would change before you publish. Nothing is charged.
- Statuses and errorsThe five statuses a moderation can have and every error the moderation endpoints return, with the HTTP status, the code and what to do next.
Something missing or unclear on this page? Write to support@useinfrared.com.