AI Engineer for Software Engineering Evaluation
About OpenTrain OpenTrain AI is the hiring and contracting organization for this role and the #1 platform for finding and building careers in AI training and data labeling. OpenTrain helps people discover cutting-edge projects, build their AI work profile, and apply in minutes. Creating an OpenTrain account is free. About AI Training and Evaluation AI training is the human side of building modern artificial intelligence. Engineers, researchers, and other contributors create examples, test model behavior, and provide feedback that helps AI systems reason, use tools, and produce more reliable results. This project focuses on evaluating AI agents through realistic software engineering tasks. Your technical judgment will help measure how well models discover information, solve problems, use development tools, and implement dependable software. The AI Engineer Role OpenTrain AI is seeking experienced AI Engineers and software engineers for next-generation AI training and evaluation projects. You will help create reinforcement learning environments that test AI models on complex software engineering tasks using Model Context Protocol tools. This is a remote contractor opportunity for approximately 15 hours per week, with a flexible schedule that may include weekends. The expected project duration is 1 to 3 months, and urgent hiring is underway.
- Compensation: $60–$120 per hour
- Contract, part-time engagement
- Workload: approximately 15 hours per week and under 20 hours weekly
- Remote work with flexible days and hours
- Output-based payment for completed tasks meeting project specifications and quality requirements
- Minimum weekly submission requirements may apply
What You'll Do
You will contribute practical software engineering expertise to development environments and evaluation systems designed for AI agents. Individual assignments may vary according to project needs, experience, and workflow.
- Fix bugs and debug complex technical issues
- Implement new features and develop maintainable software solutions
- Refactor existing codebases
- Optimize software performance and scalability
- Design reproducible development environments
- Create deterministic verification systems
- Develop golden reference solutions for AI evaluation
- Evaluate software engineering ability and effective MCP tool usage
Required Skills and Qualifications Strong software engineering expertise and practical programming experience are the primary requirements. No prior professional AI experience is required, but you should be comfortable working from technical specifications and producing reliable, maintainable code. The role requires fluent English and experience with computer code programming labeling workflows, AI software engineering training, and evaluation subject matter. Candidates should be available to collaborate effectively in a remote environment.
- Strong proficiency in C++, Python, Java, Go, TypeScript, or Rust
- Python 3, Java, Rust, C++ fundamentals, and TypeScript experience
- Deep understanding of algorithms, data structures, and performance tuning
- Demonstrated experience debugging complex software issues
- Strong experience with feature development and codebase refactoring
- Proven ability to optimize software for performance and scalability
- Excellent written and verbal communication skills
- Strong attention to detail and ability to follow technical specifications
- Intermediate experience level
- Fluent English proficiency
Preferred Experience The following experience is helpful but not required. Candidates may be considered based on the strength of their software engineering background and practical programming work.
- Experience with large-scale or distributed codebases
- Familiarity with AI or machine learning systems
- Experience with modern developer tools and APIs
- Participation in rigorous code reviews
- Knowledge of software engineering best practices
- Experience designing automated testing or verification systems
- Familiarity with Model Context Protocol or similar tool-use frameworks
Location, Selection, and Start Timeline Candidates must be based in one of the following countries: the United Kingdom, United States, United Arab Emirates, Ireland, India, South Korea, Japan, Finland, Mexico, Brazil, Australia, Austria, Canada, Belgium, Egypt, or France. The working language is English. The selection process is designed to assess technical capability and project fit. Selected candidates should be prepared to begin their first tasks within 24–48 hours after completing onboarding.
- Submit an application with your updated resume and availability for the interview
- Complete screening questions
- Complete an approximately 30-minute AI interview
- Complete a technical assessment if required
- Proceed through hiring manager review
- Complete onboarding and receive a project assignment
- Start as soon as possible if selected
Apply To This Job