Engineer - Data Engineering
WE ARE HIRING: Engineer – Data Engineering
Location: Sri Lanka Employment Type: Contract Company Industry: Software
Role Overview
We are seeking a motivated and technically skilled Engineer – Data Engineering to design, develop, and maintain scalable enterprise data solutions and cloud-based data-processing applications.
The selected candidate will be responsible for building reliable data pipelines, integrating multiple source systems, processing large volumes of data, and supporting solutions throughout the complete software-development lifecycle. The role requires strong programming skills, hands-on cloud experience, and the ability to work effectively within an Agile and DevOps environment.
Key Responsibilities
- Design and develop scalable enterprise-grade data-processing solutions.
- Build distributed and highly available data applications that support large-scale business requirements.
- Design, develop, test, and maintain data pipelines connecting multiple source systems and target platforms.
- Develop ETL and ELT workflows for batch and near-real-time data processing.
- Process, transform, and validate large volumes of structured and unstructured data.
- Develop cloud-native data solutions using Google Cloud Platform services.
- Build data-processing and orchestration workflows using tools such as Airflow and Cloud Composer.
- Work with services such as BigQuery, Dataflow, Dataproc, Datastream, Pub/Sub, Cloud Functions, and Cloud Run.
- Support data-processing solutions using AWS services such as Glue, Lambda, EMR, and Data Pipeline when required.
- Develop data-processing applications using Python, Shell scripting, and SQL.
- Design and optimize solutions using relational databases, NoSQL databases, and distributed storage engines.
- Support streaming-data applications using technologies such as Kafka, Pub/Sub, Spark, or similar tools.
- Contribute to the development of data warehouses, data marts, data lakes, and data-mesh solutions.
- Apply software-engineering best practices to ensure clean, reusable, maintainable, and high-quality code.
- Participate in code reviews, technical design discussions, and solution-architecture activities.
- Implement and maintain Continuous Integration and Continuous Delivery pipelines.
- Use code-management and automation tools such as GitHub, GitLab, Jenkins, or equivalent platforms.
- Follow DevOps principles throughout development, testing, deployment, and production support.
- Monitor application health, data-pipeline performance, and system reliability.
- Investigate and resolve data-processing failures, performance issues, and production incidents.
- Use monitoring platforms such as Datadog or equivalent tools when required.
- Collaborate with software engineers, data engineers, quality engineers, analysts, architects, and product stakeholders.
- Participate in Agile activities, including sprint planning, daily stand-ups, reviews, and retrospectives.
- Support technical alignment across team members and contribute to implementation planning.
- Maintain technical documentation, data mappings, solution designs, and operational procedures.
- Continuously evaluate new data-engineering tools, cloud services, and development practices.
Candidate Profile
- Bachelor's degree in Computer Science, Software Engineering, Information Technology, or an equivalent field.
- Minimum 1 to 2 years of experience developing enterprise-grade data-processing applications.
- Strong programming skills in Python, Shell scripting, and SQL.
- Hands-on experience processing large volumes of data.
- Experience designing and developing ETL or ELT data pipelines.
- Practical experience with relational databases, NoSQL databases, and distributed storage engines.
- Hands-on experience with Google Cloud Platform.
- Experience with GCP services such as BigQuery, Dataflow, Dataproc, Datastream, Pub/Sub, Cloud Functions, Cloud Run, and Cloud Composer.
- Familiarity with ETL and orchestration tools such as Airflow.
- Exposure to AWS data services such as Glue, Lambda, EMR, Spark, Hive, or Data Pipeline will be an advantage.
- Experience working with streaming technologies such as Kafka, Pub/Sub, Storm, or Spark Streaming will be an advantage.
- Good understanding of distributed systems, scalable architectures, and cloud-based data processing.
- Experience working within Scrum and Agile delivery environments.
- Familiarity with DevOps practices and software-development lifecycle processes.
- Experience with source-control and CI/CD tools such as GitHub, GitLab, and Jenkins.
- Exposure to application-monitoring tools such as Datadog or an equivalent platform will be an advantage.
- Understanding of data warehousing, data marts, data lakes, or data-mesh concepts will be preferred.
- Good knowledge of software-engineering standards, testing practices, and code-quality principles.
- Strong analytical, troubleshooting, and problem-solving skills.
- Strong written and verbal communication skills.
- Ability to communicate technical ideas clearly and influence implementation decisions.
- Ability to work collaboratively within cross-functional technical teams.
- Proactive, adaptable, and committed to continuous learning and professional development.
- Strong interest in building and maintaining reliable, high-performance data solutions.
Ready to take your career to new heights?
We're , your recruitment partner in Sri Lanka, and we've got an exciting opportunity for you! Our client is looking for a dynamic individual to fill the role of Engineer – Data Engineering.
Apply now and see how you can be the perfect fit for this exclusive position!
#InTalentAsia #CareerOpportunity #JobVacancy #DataEngineer #DataEngineering #GoogleCloudPlatform #PythonJobs #CloudData #ETL #ElevateYourCareer
Originally posted on