AI Analytics Workflow Evaluator
About OpenTrain OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. It helps contributors discover projects, build a professional profile, and apply for opportunities in a fast-growing field where people help shape how modern AI systems work. OpenTrain AI is recruiting for this contractor opportunity. The role is available worldwide for English-speaking contributors. About AI Training and Data Evaluation AI training is the human side of building artificial intelligence. Contributors review, test, annotate, and evaluate AI outputs so models can produce more accurate and useful results. This work offers remote, flexible opportunities for people with technical and specialist expertise.
- Work remotely from anywhere with an internet connection
- Contribute to the development of cutting-edge AI systems
- Build experience in evaluation, quality assurance, and data-focused AI workflows
The Role As an AI Analytics Workflow Evaluator, you will assess how AI assistants perform realistic data-analysis workflows connected to cloud data warehouses. You will check whether generated figures, SQL queries, joins, filters, and time windows are accurate and logically sound. The work combines hands-on data analysis, Snowflake environment management, product-behavior investigation, and rigorous evaluation of AI-generated analytical results. The opportunity is classified as entry level, while advanced SQL and Snowflake capabilities are important for success.
- Contractor, part-time engagement
- Worldwide opportunity
- English-language work
- Time requirement: 20+ hours per week
- Pay: $50–$60 USD per hour
What You'll Do
You will execute structured evaluation scenarios and compare AI-generated outputs with source data. Your findings will help identify subtle analytical errors and improve the reliability of AI assistants working with business data.
- Evaluate anomaly investigations, KPI reports, and data-refresh workflows
- Write and run SQL queries to verify AI-generated results against source data
- Audit figures, joins, filters, time windows, aggregations, and analytical logic
- Maintain, reset, and manage seeded Snowflake datasets for accurate, reproducible scenarios
- Administer warehouse roles, permissions, grants, and access controls
- Configure and document connectivity and authentication across analytics and SaaS platforms
- Investigate undocumented product behavior and record findings clearly
- Participate in calibration sessions to keep grading and scoring consistent
- Review AI-generated SQL and analytical outputs using structured evaluation criteria
Required Skills and Experience Advanced SQL skills and expertise with Snowflake warehouses are important for this role. You should be able to reconcile reported metrics with raw data, identify aggregation or logic errors, and make sound judgments when evaluating analytical results. Strong written and verbal communication, careful documentation, and experience with rubric-based evaluation or quality assurance are required or highly relevant. At least three years of hands-on experience as a data analyst or analytics engineer is preferred.
- Advanced SQL skills for validating AI-generated analytical results
- Snowflake expertise, including warehouses, query history, roles, grants, and access controls
- Ability to reconcile reported metrics with raw data
- Ability to identify subtle aggregation and logic errors
- Experience administering database access and security permissions
- Understanding of KPI definitions and business or finance reporting practices
- Strong written communication, verbal communication, and documentation skills
- Sound judgment for rubric-based evaluation and quality assurance
Preferred Background Experience working with AI assistants and analytics integrations will help you contribute effectively. Familiarity with product connectors, authentication, and workplace software is also useful for investigating complete analytics workflows.
- Experience using Claude or ChatGPT connectors for analytics workflows
- Experience reviewing AI-generated SQL or analytical outputs
- Background in rubric-based evaluation, quality assurance, or data labeling
- Familiarity with OAuth and security integrations
- Familiarity with Slack, Google Workspace, or Microsoft 365
- Experience as a data analyst or analytics engineer
Build Your AI Training Career OpenTrain gives freelancers a place to build a lasting record of AI training and data-labeling experience. By creating a profile and contributing to specialized evaluation work, you can showcase relevant expertise, discover projects that match your skills, and grow a portfolio in the expanding AI industry.
- Create an OpenTrain account for free
- Build a profile that reflects your technical and evaluation experience
- Find opportunities aligned with analytics, SQL, Snowflake, and AI evaluation skills