Remote Data Annotation Specialist

Remote Data Annotation Jobs in Los Angeles Rex.zone is hiring a Remote Data Annotation Specialist to support training data creation and evaluation for modern AI systems. You will label and review text, images, audio, and conversations used in LLM training pipelines, RLHF workflows, and model evaluation. This is a full-time, remote-first role supporting distributed teams.

What You Will Do

Perform data labeling for NLP, computer vision annotation, and multimodal (text-image) tasks Execute RLHF-style ranking and preference judgments to improve model alignment Run LLM evaluation tasks (rubric-based scoring, prompt-response grading, error categorization) Apply content safety labeling policies (toxicity, self-harm, harassment, sensitive attributes) Conduct QA evaluation and audits; track issue/disagreement rates and improve consistency Follow detailed annotation guidelines, document edge cases, and escalate ambiguous cases Workflow and Tools Work in web-based annotation platforms using task instructions, gold-standard examples, and calibration sets. Maintain review notes and support continuous improvement of training data quality. Helpful familiarity: inter-annotator agreement, rubric design, labeling taxonomies, and dataset versioning.

Required Qualifications

Mid-Senior experience in data annotation, data labeling, QA evaluation, or model evaluation Strong attention to detail and consistent guideline-based decision-making Comfort working with ambiguity; strong written communication for documenting edge cases Reliable remote work habits and time management Preferred Qualifications RLHF / preference ranking experience LLM evaluation and prompt evaluation experience Named entity recognition (NER) or taxonomy-based labeling Computer vision annotation (bounding boxes, segmentation, keypoints) Quality programs: auditing, calibration sessions, disagreement analysis Why Rex.zone Contribute to measurable improvements in training data quality and model evaluation outcomes across domains including NLP, computer vision, LLM training, and content safety in a remote-first environment. How To Apply Apply via Rex.zone with a resume highlighting annotation experience, QA evaluation work, and any RLHF, LLM evaluation, NER, computer vision annotation, or content safety labeling projects.