Remote AI Data Annotation Jobs in New York
Overview
Remote AI Data Annotation roles at Rex.zone focus on training data quality for modern AI systems, including data labeling, RLHF evaluation, prompt evaluation, and QA review across NLP and computer vision workflows.
Key Responsibilities
- Produce high-accuracy labels for text, image, and multimodal datasets aligned to project schemas
- Execute RLHF tasks such as pairwise preference judgments, ranking, and rationale writing when required
- Complete prompt evaluation and response grading for helpfulness, correctness, and policy alignment
- Perform QA evaluation, including peer review, spot checks, and adjudication with clear decision logs
- Apply named entity recognition (NER) and taxonomy tagging for NLP datasets
- Support computer vision annotation (bounding boxes, polygons, keypoints, segmentation masks) per spec
- Conduct content safety labeling using defined policies, thresholds, and escalation paths
- Track throughput and quality metrics and maintain strict annotation guidelines compliance
Required Qualifications
- Experience in AI data annotation, data labeling, QA evaluation, or related data operations
- Ability to follow detailed guidelines and work within rubric-based evaluation
- Familiarity with NLP concepts (e.g., NER, text categorization)
- Strong attention to detail, confidentiality, and comfort with repetitive work
Tools & Workflow Work is completed online in web-based labeling and evaluation tools with task queues, guideline wikis, gold checks, calibration, and reviewer feedback loops. Some projects may involve sensitive content for content safety labeling. How To Apply Apply on Rex.zone and complete any required skills checks or calibration tasks for production labeling workflows. Apply To This Job