[Remote] Quality Analyst

Note: The job is a remote job and is open to candidates in USA. Microsoft is seeking a detail-oriented and customer-focused Content QA Analyst to ensure the quality, reliability, and effectiveness of content experiences across Microsoft 365. The role develops test plans, executes manual and automated tests, evaluates AI-generated outputs, tracks defects, and provides recommendations to improve content and customer experience.

Responsibilities

• Develop and execute test plans for templates, prompt templates, AI-generated content, and other Microsoft 365 content experiences
• Perform manual and automated testing across Word, Excel, PowerPoint, Create, and Copilot experiences
• Validate content quality, functionality, usability, accessibility, and adherence to design fidelity
• Identify, document, reproduce, and track defects through resolution
• Evaluate AI-generated outputs against defined quality standards and acceptance criteria
• Assess content accuracy, relevance, usability, formatting, and customer experience
• Provide clear recommendations to improve model and content performance
• Create and maintain test scripts, test cases, validation frameworks, and execution documentation
• Develop repeatable testing approaches for new content experiences and features
• Partner with content and cross-team stakeholders to identify opportunities for automation
• Analyze testing results and communicate findings to stakeholders
• Identify patterns, recurring issues, and opportunities for quality improvements
• Track testing outcomes and ensure changes are reflected in documentation and content production workflows

Skills

• 2-4 years of experience in QA, content testing, usability testing, or quality assurance
• Bachelor's degree in Content Automation, Computer Science, Engineering, Information Systems, or related field
• Experience creating test plans, test cases, and test reporting documentation
• Experience scripting in Python, PowerShell, or similar languages
• Knowledge of Microsoft 365 products and productivity workflows
• Strong understanding of testing methodologies and quality processes
• Experience testing AI-generated content or generative AI experiences
• Understanding of design principles, content quality, and user experience
• Experience working with automation frameworks
• Familiarity with human evaluation methodologies and quality rubrics
• Excellent analytical and problem-solving skills
• Strong written and verbal communication skills

Benefits

• Medical, dental & vision
• Hospital plans
• 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available
• Life Insurance (Company paid Basic Life and AD&D as well as voluntary Life & AD&D for the employee and dependents)
• Company paid short and long-term disability
• Health & Dependent Care Spending Accounts (HSA & DCFSA)
• Employee Assistance Program
• Time Off/Leave (PTO, Allegis Group Paid Family Leave, Parental Leave)

Company Overview