Data Engineers

Role: Data Engineer – SQL to Microsoft Fabric Modernization/Migration Location: 100% remote This is a 10-week project (that is about 2.5 months) – candidates MUST be ok with that duration Targeting October 12th or 19th as a start date This is the formal job description: We are seeking a Data Engineer to support a Microsoft Fabric modernization initiative focused on migrating existing on-premises SQL Stored Procedures workloads into a Fabric Lakehouse environment. This role will be responsible for converting SQL/T-SQL stored procedures into Fabric notebooks, building and orchestrating data pipelines, validating migrated datasets, and creating technical documentation to support ongoing operations. This position will primarily support the Silver Layer within a Medallion Architecture and will play a critical role in ensuring data accuracy, consistency, and scalability throughout the migration effort. Team structure is small, must be highly motivated and self-motivated. Will have access to some SMEs but this role is primarily responsible for the vast majority of the work here. Must be comfortable working in a small team with one other to deliver fast and quick with great quality. Key ResponsibilitiesConvert existing SQL/T-SQL stored procedures into Microsoft Fabric Notebooks using PySpark and Spark SQL. Design and build the Silver Layer of a medallion architecture in Fabric using Lakehouse, ensuring parity between Fabric data structures and existing on-premises SQL environments. Develop and maintain orchestration frameworks using Microsoft Fabric Data Pipelines. Build automated validation and reconciliation processes to ensure migrated data matches source systems on a 1:1 basis. Create reusable testing frameworks and validation artifacts to continuously verify data integrity and accuracy. Support data pipeline monitoring, scheduling, and orchestration activities within Microsoft Fabric. Partner with architects, engineers, and business stakeholders to ensure successful migration outcomes. Develop and maintain technical documentation detailing solution design, implementation approaches, validation procedures, and operational support processes. Analyze and troubleshoot data issues across Fabric notebooks, pipelines, and Lakehouse objects. Required Qualifications 5-7+ years of experience as a Data Engineer or similar role. 1-2+ years of hands-on experience with Microsoft Fabric. Strong expertise in SQL and T-SQL, including: Stored procedures Query optimization Data transformations SQL-based business logic Experience migrating or translating SQL-based processing into modern cloud-based data platforms. Strong hands-on experience with: PySpark Spark SQL Microsoft Fabric Notebooks Experience developing and managing data pipelines and orchestration workflows. Deep understanding of: Lakehouse architecture Microsoft Fabric data flows Data Pipelines Medallion Architecture concepts Experience building automated data quality, reconciliation, and validation frameworks. Strong technical documentation and communication skills.

Preferred Qualifications

Experience working with metadata-driven frameworks. Experience implementing orchestration and scheduling strategies within Microsoft Fabric. Familiarity with enterprise data migration initiatives. Experience supporting cloud-based analytics and modern data platform solutions. Technical Environment Microsoft Fabric Fabric Notebooks Fabric Data Pipelines Lakehouse Architecture Medallion Architecture (Silver Layer Focus) SQL / T-SQL PySpark Spark SQL