[Remote] Quality Analyst (Remote)
Note: The job is a remote job and is open to candidates in USA. Hire Feed is hiring on behalf of one of its clients for an AI Quality Analyst - English. The role evaluates personalized AI-generated responses, designs multi-turn prompts, compares model outputs, and provides detailed feedback to improve AI personalization performance.
Responsibilities
- Evaluate the quality of personalized responses from an AI model using your own personal experiences as context
- Design creative, multi-turn prompts that test the model’s ability to integrate personal data sources accurately
- Assess model responses for grounding, integration, and helpfulness, identifying incorrect personalization or forced connections
- Review side-by-side comparisons of AI responses to detect subtle differences in relevance and accuracy
- Provide detailed feedback on model behavior to improve personalization performance
Skills
- English proficiency with a high degree of competence in reading and writing
- Experience in prompt engineering or designing test cases for AI models
- Ability to analyze nuanced and ambiguous AI responses with analytical rigor
- Familiarity with personalization concepts and evaluation metrics in AI systems
- Attention to detail to identify subtle discrepancies in model outputs
- Willingness to use your primary personal Google account and enable personal data sources for assessment
- Full-time availability in your local time zone to support a global, 24-hour operations team
Benefits
- Remote work arrangement; work from anywhere
- Full-time role
Company Overview