[Remote] Data Engineer

Note: The job is a remote job and is open to candidates in USA. Pyx Health is a healthcare technology company developing solutions supported by data and analytics. The Healthcare Data Engineer will build and maintain Azure-based data infrastructure, develop reliable pipelines and Airflow workflows, and implement data models, quality monitoring, and governance practices while collaborating with technical and business teams.

Responsibilities

• Build and maintain data pipelines for ingesting, transforming, and cleaning healthcare data in the Azure cloud using Databricks, PySpark, and Delta Lake
• Implement pipeline logic from defined specifications, with guidance from senior engineers on architectural decisions
• Build reusable testing frameworks to ensure pipeline reliability
• Monitor pipelines for failures and performance issues, escalating complex problems appropriately
• Develop and maintain Airflow DAGs with appropriate error handling and retry logic
• Support deployment and configuration of pipelines via Astronomer on Azure using the Astro CLI
• Contribute to improving pipeline reliability and reducing manual intervention
• Implement data models for efficient storage and retrieval using Delta Lake, including merge/upsert patterns
• Design and optimize Delta Lake tables for analytic workloads
• Leverage Unity Catalog for data governance, organization, and security
• Apply Change Data Capture (CDC) patterns under the direction of senior team members
• Write and optimize T-SQL and Databricks SQL scripts, stored procedures, and ETL support
• Work within the Azure ecosystem including ADLS Gen2, Key Vaults, Logic Apps, and Azure DevOps
• Follow established security and scalability standards when building data infrastructure
• Support infrastructure tasks with guidance from senior engineers on architecture
• Ensure scripts and datasets are well-documented to support enterprise data governance
• Design, implement and continuously improve automated data quality monitoring, reconciliation, and alerting to ensure reliable downstream reporting
• Troubleshoot and resolve pipeline failures, documenting root causes and resolutions
• Work closely with data scientists, analysts, and business stakeholders to understand reporting requirements, troubleshoot issues, and improve data assets
• Participate in code reviews, incorporating feedback to improve code quality
• Document pipelines, processes, and implementation decisions clearly and consistently
• Stay current with data engineering technologies, particularly in the healthcare space

Skills

• Only candidates residing in the USA may apply
• •       2–4 years of experience as a Data Engineer or in a closely related data role
• •       Hands-on experience with Azure cloud services (ADLS, Databricks, or similar)
• •       Working knowledge of SQL for scripting and data modeling including T-SQL, Spark, and Databricks SQL
• •       Ability to contribute to technical projects with moderate oversight
• •       Strong communication skills — comfortable asking questions and providing updates to cross-functional partners
• •       Familiarity with CI/CD concepts, Git, and version control workflows
• •       Solid problem-solving skills with a systematic approach to debugging
• •       Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field (or equivalent work experience)
• •         Databricks SQL (Mid):Delta Lake fundamentals, basic merge/upsert patterns, familiarity with CDC concepts
• •         Python (Mid):Pipeline logic, data transformation, SQL scripting; some PySpark/Spark DataFrame experience
• •         Databricks Spark Notebooks (Mid):Notebook-based development, basic cluster usage, Delta table operations
• •         Airflow Python Development (Mid):Ability to write and maintain DAGs; understanding of error handling and retry patterns
• •         Airflow Astro Configuration (Foundational):Familiarity with Astronomer or willingness to learn; basic Astro CLI usage
• •         Azure Ecosystem (Foundational):Working knowledge of ADLS Gen2, Key Vaults, and Azure DevOps
• •         T-SQL (Foundational–Mid):Basic stored procedures, SQL Server querying, and data manipulation
• •       Azure Data Factory (Foundational):Basic familiarity with pipeline authoring and triggers
• •       Git + Azure DevOps CI/CD (Foundational):Experience with branching, pull requests, peer review processes, and version control best practices as well as deploying production data pipelines using CI/CD workflows
• •       Familiarity with healthcare data standards and regulations (HIPAA, HL7, etc.) is a plus

Company Overview