Instruction Following Data Labeler

Pay: $20–35/hour

You'll label text data focused on instruction-following behavior — specifically, evaluating whether a model's output correctly satisfies the constraints and intent of a given user instruction. Tasks include binary labels (did the model follow the instruction — yes/no), multi-attribute scoring (tone, format, completeness, constraint adherence), and span-level annotation (highlight exactly which part of the output fails to follow a stated constraint). You'll work with a detailed labeling guide and per-project instructions that define edge cases explicitly.

The data covers a wide range of instruction types: formatting requests ("respond only in bullet points"), content constraints ("do not mention competitors"), persona instructions, length targets, and structured output requirements (JSON, tables, lists). Each task is self-contained — you read the instruction, read the output, apply the rubric, and submit. Throughput targets are typically 30–50 tasks per hour for straightforward binary tasks, lower for multi-attribute annotation. Work is async and batched.

You should be a strong reader with high attention to detail and the ability to apply written guidelines consistently across varied examples. Prior experience in content moderation, linguistic annotation, or data quality work is directly transferable. You don't need a technical background — but you do need to read instructions carefully and flag ambiguous cases through the proper escalation channel rather than guessing.

Skills & requirements

• Instruction interpretation
• Rubric-based text labeling
• Span-level annotation
• Constraint adherence evaluation
• Attention to detail
• Annotation guideline compliance

Category: Data Labeling