Remote Data Annotator Jobs in New York
About The Role As a Remote Data Annotator supporting New York-based programs, you will label and evaluate datasets used to train and validate machine learning models. Your work improves training data quality, supports RLHF feedback loops, and drives model performance improvement across NLP, computer vision, and LLM workflows.
What You Will Do
- Produce high-accuracy labels for text, images, and multimodal inputs
- Perform RLHF-style preference ranking and prompt evaluation for LLMs
- Complete QA evaluation checks, audits, and consistency reviews
- Apply named entity recognition and taxonomy tagging
- Conduct content safety labeling for policy and harm categories
- Document edge cases and guideline gaps to improve labeling instructions
- Collaborate asynchronously with reviewers, QA, and data operations teams
Core Workflows You May Support
- Large language model evaluation and prompt-response grading
- Classification and ranking tasks; training data curation
- Computer vision annotation (bounding boxes, polygons, keypoints)
- Conversation quality scoring and safety policy enforcement
Required Qualifications
- Professional experience in data annotation or data labeling
- Strong written English comprehension and attention to detail
- Ability to follow detailed guidelines with high precision and consistency
Preferred Qualifications
- Experience with RLHF, LLM training pipelines, prompt evaluation, or QA evaluation
- Familiarity with NER, content safety labeling, and CV annotation tools
Remote Work Notes (New York) These roles are Remote and can be performed from New York. Some projects may require availability aligned with US business hours or structured QA review cycles.