[Remote] GCP Data Engineer
Note: The job is a remote job and is open to candidates in USA. CoSourcing Partners is an enterprise AI and IT services company seeking a GCP Data Engineer to design, develop, and optimize scalable data solutions on Google Cloud Platform. The role focuses on batch and streaming pipelines, modern data architectures, data quality, ETL/ELT modernization, governance, and enabling analytics, reporting, and machine learning use cases.
Responsibilities
- Design, build, and optimize scalable batch and real-time (streaming) data pipelines using GCPnative services
- Develop and maintain data ingestion frameworks leveraging tools such as Pub/Sub, Dataflow, and Cloud Storage
- Implement data transformation pipelines using BigQuery, dbt, and Python-based workflows
- Ensure efficient handling of large-scale structured and unstructured datasets
- Design and implement high-performance data models for cloud-based data lakes, data warehouses, and analytics platforms
- Optimize data schemas and partitioning strategies in BigQuery for performance and cost efficiency
- Support modern architectures such as medallion (bronze/silver/gold) layers and lakehouse patterns
- Write advanced SQL queries for transformation, validation, and analytics
- Develop scalable data processing logic using Python and/or Apache Beam
- Build reusable, modular, and maintainable code for data workflows
- Implement and maintain data quality checks, validation rules, and anomaly detection frameworks
- Enable data observability through monitoring, logging, and alerting mechanisms
- Ensure highly reliable data pipelines with fault tolerance and error handling strategies
- Support migration and modernization efforts from legacy ETL tools (e.g., Talend) to GCP-native ELT frameworks (dbt)
- Optimize existing pipelines for performance, scalability, and maintainability in cloud environments
- Drive adoption of ELT best practices using BigQuery as the compute engine
- Collaborate with data architects, business analysts, and machine learning teams to deliver trusted datasets
- Translate business requirements into scalable data solutions
- Provide technical guidance and support for downstream analytics and reporting use cases
- Drive adoption of best practices in cloud data engineering, CI/CD, and DevOps
- Implement secure data access controls using IAM roles, policies, and governance frameworks
- Follow standards for code quality, version control (Git), and automated deployments
Skills
- Bachelor's or Master's degree in Computer Science, Engineering, or related field
- 4+ years of experience in data engineering or data platform development
- Hands-on experience with Google Cloud Platform (GCP) services:
- BigQuery
- Dataflow
- Pub/Sub
- Cloud Storage
- Strong proficiency in SQL and Python
- Experience with dbt (Data Build Tool) or similar ELT frameworks
- Experience building batch and streaming data pipelines
- Strong problem-solving and analytical skills
- Ability to work in cross-functional teams
- Excellent communication and documentation skills
- Focus on performance optimization and scalability
- Attention to data quality and reliability
- Experience with Apache Beam or Spark
- Familiarity with Talend or other ETL tools and migration to cloud-native solutions
- Knowledge of data lakehouse architectures and modern data stack
- Experience with CI/CD tools (e.g., GitHub Actions, Cloud Build, Jenkins)
- Understanding of data security, governance, and compliance standards
- Exposure to machine learning data pipelines and feature engineering
Company Overview
Company H1B Sponsorship