[Remote] ML Engineer - Model Evaluation Expert
Note: The job is a remote job and is open to candidates in USA. Mercor connects elite creative and technical talent with leading AI research labs. The company is seeking a Machine Learning Engineer to design research-based tasks, run and analyze training experiments, evaluate frontier models, and collaborate with researchers on rigorous and fair task development.
Responsibilities
- Design tasks by transforming real ML research ideas into well-defined, multi-step tasks
- Run experiments by implementing changes, executing training experiments, and analyzing results to define correct solutions
- Explore reinforcement learning concepts such as reward functions and training behavior in task development
- Evaluate frontier models' performance on tasks and identify areas of improvement
- Collaborate with researchers to ensure tasks are consistent, rigorous, and fair
Skills
- • MSc or PhD in machine learning, computer science, or a related STEM field
- • 1+ years in a research or research-engineering role
- • Experience in training and evaluating ML models and conducting end-to-end experiments
- • Proficiency in Python and Git
- • Strong understanding of large language models and their evaluation
- • Basic knowledge of reinforcement learning
- • Experience in AI training, model evaluation, or benchmark/task authoring
Benefits
- Remote work
- Paid weekly
Company Overview
Company H1B Sponsorship