[Remote] Data Engineering (DE) Lead
Note: The job is a remote job and is open to candidates in USA. TechDoQuest is seeking a Data Engineering Lead to own data-engineering delivery on GCP and DataFlow. The role is responsible for designing and implementing ingestion, transformation, reconciliation, publication, monitoring, CI/CD, and operational engineering standards across large-scale data pipelines.
Responsibilities
- Define reusable DataFlow/GCP ingestion and transformation patterns
- Implement source ingestion from Epic, ELMS, Oracle, terminology and other approved sources
- Build raw/source-preserving, standardised/cleansed and curated/mastered processing layers
- Implement standardisation, validation, transformation and reference-data enrichment
- Build reconciliation controls between source, MDM/RDM and downstream publication
- Build publication pipelines to GCP data products, APIs, events and tables/files
- Implement replay, restart, idempotency, audit and monitoring; optimise performance and cost
- Establish CI/CD, code quality and production support procedures
Skills
- • 8+ years in data engineering, hands-on with GCP and DataFlow (or equivalent)
- • Large-scale batch, incremental and event-driven pipeline experience
- • Experience feeding and reconciling MDM/RDM data
- • Strong on operationalisation: monitoring, replay, CI/CD
- • BigQuery, Pub/Sub and Dataflow depth
- • Healthcare data engineering
- • Integration with Informatica MDM loads
Company Overview
Company H1B Sponsorship