[Remote] Data Engineer (Microsoft Azure)
Note: The job is a remote job and is open to candidates in USA. Luxoft is seeking a Data Engineer to design and deliver scalable cloud-based data engineering solutions on Microsoft Azure. The role focuses on building and optimizing ETL/ELT pipelines, integrating structured and unstructured data, developing data models and AI-based solutions, and supporting analytics, machine learning, reporting, and data products. The engineer will also contribute to data architecture and standards while mentoring other data engineers.
Responsibilities
- Design, develop, and optimize ETL/ELT solutions using Azure Databricks, LakeBase, and Spark
- Build and deploy large-scale data pipelines supporting analytics, machine learning, predictive modeling, reporting, and data products
- Integrate structured and unstructured data from databases, APIs, FTPs, cloud storage, and various file formats
- Develop and optimize complex SQL queries and data models
- Build AI-based solutions for business needs, process automation, and operational efficiency
- Improve data reliability, integrity, performance, and overall platform quality
- Contribute to data architecture, engineering standards, frameworks, and best practices
- Provide technical guidance and mentorship to other data engineers
Skills
- 7+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes
- 5+ years of hands-on Python experience and 2+ years of experience with SQL, complex data schemas, and query performance optimization
- At least 2 years of hands-on experience with Redis-backed state management and event-driven processing, as well as building cloud solutions using Azure Databricks with Unity Catalog, Snowflake, Azure Functions, and Service Bus
- Practical experience with Terraform-based DevOps automation and CI/CD tools including GitHub Actions, Git, Artifactory, and Sonar is required
- Azure Databricks
- Apache Spark
- Python
- SQL
- ETL/ELT
- Data Engineering
- Data Modeling
- Redis
- Snowflake
- Azure Functions
- Azure Service Bus
- Terraform
- GitHub Actions
- Git
- Artifactory
- Sonar
- Excellent communication skills
- Bachelor's degree in Computer Science, Engineering, Mathematics, or a related discipline
- Strong understanding of Data Lake, Delta Lake, and Data Warehousing architectures, as well as advanced data modeling techniques including slowly changing dimensions, aggregation, partitioning, and indexing
- Ability to independently troubleshoot and performance-tune large-scale enterprise systems, collaborate effectively across technical and business teams, mentor engineers, and communicate technical concepts to both technical and non-technical audiences
- Exposure to LLM/AI solution delivery is preferred
Company Overview
Company H1B Sponsorship