Data Engineer
Position Overview We are seeking an experienced Data Engineer to join an advanced research team dedicated to developing and refining next-generation, large-scale multimodal AI models. In this high-priority role, you will bridge the gap between human evaluation workflows and robust data pipeline development. You will work directly with research scientists and software engineers to design human studies, manage data annotation processes, build evaluation interfaces, and create automated pipelines that directly measure and improve model performance. Key Responsibilities 1. Data Collection & Human Evaluation (~50% Effort) Study & Workflow Design: Design, deploy, and manage human evaluation studies and annotation workflows to assess multimodal model behaviors. Interface & Tooling Development: Build and maintain front-end annotation UIs and custom tooling for human raters. Quality Operations & Rater Management: Oversee day-to-day operations with internal and external vendor rating teams across multiple time zones. Monitor evaluation quality, establish feedback loops, and deliver quality control checks. 2. Pipeline Automation & Analytics Engineering (~50% Effort) Data Pipeline Construction: Architect and maintain efficient data processing pipelines, data visualization tools, and performance analysis frameworks. Workflow Automation: Drive efforts to eliminate manual and repetitive data tasks, aiming for 80%–90% automation in data allocation and ingestion workflows. Model-Data Analytics: Develop diagnostic tools and visual dashboards to analyze how dataset changes impact model behavior across various scenarios.
Required Qualifications
Core Software Engineering: Advanced proficiency in Python with expertise in data science and manipulation libraries (e.g., Pandas, NumPy). Data Pipeline Engineering: Proven hands-on track record in building, automating, and deploying production-grade data pipelines. Human-in-the-Loop Workflows: Demonstrable experience building annotation UIs and designing human study/evaluation frameworks.
Preferred Qualifications
Generative AI Familiarity: Working knowledge of large AI model evaluation, multimodal datasets, and model behavior diagnostic methods. Cloud & Modern Tooling: Familiarity with cloud platforms and modern workflow automation or AI agent frameworks. Academic Background: An advanced degree (Master’s or Ph.D.) in Computer Science, Data Science, or a related quantitative field is a plus, though practical engineering capability is prioritized. Work Environment & Location Requirement Location Restriction: Candidates must reside and work remotely from Oregon . Collaboration: High-touch collaboration environment working alongside researchers and engineers. Daily code reviews and frequent communication with globally distributed annotation teams are expected.