Remote Data Annotation Jobs Amsterdam
Remote Data Annotation Jobs Amsterdam (Remote) Rex.zone connects experienced annotators with real-world AI/ML training workflows, including data labeling, RLHF evaluation, prompt evaluation, and training data quality improvements for large language models. This is a full-time remote role supporting distributed teams and rubric-driven annotation operations. About The Role As a mid-senior Data Annotation professional, you will deliver high-precision labels and human judgments used to train, fine-tune, and evaluate AI systems. Work spans NLP classification, named entity recognition, prompt evaluation for LLMs, RLHF preference ranking, computer vision annotation, and content safety labeling.
Key Responsibilities
- Annotate and review datasets for NLP, CV, and content safety tasks while following detailed rubrics
- Perform RLHF workflows (preference ranking, pairwise comparisons, justification capture)
- Execute QA evaluation: sampling plans, discrepancy analysis, and rework loops to reduce label noise
- Document edge cases and propose guideline updates to improve inter-annotator agreement
- Support prompt evaluation and model evaluation sets measuring helpfulness, correctness, and safety
- Maintain secure handling of sensitive content and adhere to confidentiality requirements
Required Qualifications
- 3+ years in data annotation, data labeling, QA evaluation, or model evaluation for AI/ML
- Experience with annotation guidelines compliance and quality calibration methods
- Familiarity with RLHF, prompt evaluation, or LLM evaluation concepts
- Strong analytical writing for issue reports and edge-case documentation
- Comfort working remotely with distributed teams and structured throughput/quality metrics
Compensation
Hourly rate: $30–$50/hr (USD), depending on scope and evaluation responsibilities.