Browse documentationQuality safeguards
Protect the queue without exposing reviewer identities or answer keys
GenMedia combines database idempotency, server-selected attention controls, server-timed playback, coarse device binding, and private risk review. The reviewer receives useful guidance, while answer keys, device digests, exact thresholds, and signal-level diagnostics remain restricted.
| Control | Reviewer experience | Server boundary |
|---|---|---|
| Submission idempotency | A retry resumes safely | One assignment has one accepted decision and at most one reward |
| Attention control | One Session item may ask for a clearly named option | The challenge item and expected token are selected and stored privately by the server |
| Playback requirement | Progress appears for every required candidate | Only bounded, ordered, unique timeline coverage is credited |
| Guest evidence weighting | A guest sees no private quality score | Completed guest evidence contributes up to 70 percent through a 0.70 base and private terminal behavior-quality multiplier |
| Shared-network cohort | Multiple allowed guests can finish and save | Only the first task plus network and user-agent anonymous cohort can contribute provisional guest evidence |
| Device binding | No fingerprint or device identifier is shown | A random first-party cookie is HMAC-bound; raw cookie, IP, user agent, and exact dimensions are never stored in the risk tables |
| Quality state | A generic guidance notice may appear before access is limited | Detailed signals and review actions are available only to authorized operators |
The reviewer brief comes first
Before a Session starts, the reviewer sees the task owner’s title, question, description, introduction, criteria, media requirement, and any Reviewer Practice or Golden Control prerequisites. The reviewer must acknowledge that they will inspect the required media, preserve the blind, and submit their own independent judgment.
This acknowledgement is stored with the Session. It does not replace the Terms, the device and media preflight, or a prerequisite module.
Attention checks do not reveal the answer contract
The server chooses a challenge position inside the Session and returns only the prompt and opaque choices for that one assignment. The expected answer remains private. A replay or repeated submit cannot create another scored challenge or another reward.
A failed attention check is recorded as a quality signal and the associated decision does not earn fal credit. The public result and export paths never include the private answer key.
Warnings are useful; cutoffs are quiet
When recent evaluation quality first needs attention, the reviewer receives a generic notice asking them to reread task instructions, complete required playback and attention checks, and submit only independent judgments. The notice does not expose a score, signal, threshold, or other reviewer’s behavior.
If the private quality state later requires a pause or manual review, new assignments stop without sending a cutoff notification or revealing the rule. An authorized admin can inspect the bounded report and record a clear, temporary pause, or manual-review action.
Guest evidence is useful but quality-adjusted
A completed unclaimed guest decision can enter official analysis and Elo at up to 70 percent influence: a 0.70 base weight multiplied by a private terminal behavior-quality factor. Failed attention controls, severe automation evidence, anomalous decision speed, or blind-slot patterns can reduce the final influence, including to zero.
The browser and public exports never receive the private score, threshold, signal value, or rule timing. They expose only the safe result state needed to explain whether evidence was included.