Data Engineer

Job Title: Data Engineer Scope of Services The selected candidate will be responsible for: - Assessing feasibility and technical requirements for LINKS → DataLake integration. - Collaborating with OPH Immunization Program, OPH Bureau of Health Informatics, and STChealth on data specifications and recurring ingestion pipelines. - Building and optimizing ETL workflows for LINKS and complementary datasets (e.g., Vital Records, labs, registries). - Designing scalable data workflows to enhance data quality, integrity, and identity resolution. - Implementing data governance, observability, and lineage tracking across all pipelines. - Mentoring engineers, supporting testing activities, and enforcing best practices in orchestration and architecture. - Documenting and communicating technical solutions to both technical and non-technical stakeholders. Mandatory Expertise & Experience - 3 years of experience in data engineering and/or data architecture. - 2 years of experience with Python for ETL and automation (pandas, requests, API integration). - 2 years of hands-on experience with SQL queries, stored procedures, and performance tuning (Oracle, SQL Server, MySQL preferred). - 1 year of experience with ETL orchestration tools (e.g., Prefect, Airflow, or equivalent). - 1 year of experience with cloud platforms (Azure, AWS, or GCP), including data onboarding/migration. - 1 year of exposure to data lake / medallion architecture (bronze, silver, gold). - 2 years of experience preparing written documentation and conducting verbal communication for cross-functional collaboration. Preferred (Not Mandatory) Expertise & Experience - 5+ years of data engineering experience. - Experience integrating or developing REST/JSON or XML APIs. - Familiarity with CI/CD pipelines (GitHub Actions, Azure DevOps, etc.). - Exposure to Infrastructure as Code (Terraform, CloudFormation). - Experience with data governance and metadata tools (Atlan, OpenMetadata, Collibra). - Experience with public health or healthcare datasets, including PHI/PII handling. - Familiarity with SAS and R workflows for supporting epidemiologists and analysts. - Experience with additional SQL platforms (Postgres, Snowflake, Redshift, BigQuery). - Familiarity with data quality frameworks (Great Expectations, Deequ). - Experience with real-time/streaming tools (Kafka, Spark Streaming). - Familiarity with big data frameworks for large-scale transformations (Spark, Hadoop). - Knowledge of data security and compliance frameworks (HIPAA, SOC 2, etc.). - Experience working in an Agile/Scrum environment. Notes - Lumen and / or its clients will not provide equipment (Laptop, monitor, etc.) to the selected contractor. The contractor must have their own equipment. - Access to a virtual desktop set up (software) will be provided by Lumen’s client, allowing the user access to the required systems and technology. - Lumen Solutions Group Inc. is an Equal Opportunity Employer and does not discriminate in employment on the basis of Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.