[Remote] Data Platform Engineering Lead
Note: The job is a remote job and is open to candidates in USA. ClifyX is a company seeking a highly skilled Data Platform Engineering Lead to architect, build, and manage enterprise-scale cloud-native data platforms. The role involves leading platform engineering, infrastructure automation, and ensuring operational excellence across the Enterprise Data Office (EDO) while supporting Asset Management business functions.
Responsibilities
- Lead the design, implementation, and management of enterprise data platforms
- Define platform engineering strategy aligned with EDO and enterprise architecture standards
- Establish reusable platform frameworks, accelerators, and engineering best practices
- Drive cloud-native modernization initiatives across data and analytics ecosystems
- Own platform lifecycle management, scalability, availability, and reliability
- Lead enterprise-wide Databricks platform architecture and administration
- Design and implement Lakehouse architecture leveraging Delta Lake
- Manage: Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, MLflow, Structured Streaming, Cluster Policies, Security and Access Controls
- Optimize platform performance and cost efficiency
- Build governance controls across development, test, and production environments
- Lead cloud engineering activities using AWS services: S3, ECS, EKS, Lambda, Glue, Cloudwatch
- Infrastructure as a Code (IaC)
- Build reusable Terraform modules
- Automate environment provisioning
- Implement infrastructure standardization and governance
- Maintain infrastructure version control
- Ensure compliance through Infrastructure-as-Code practices
- Design and implement CI/CD pipelines
- Automate deployment and release management processes
- Enable Infrastructure-as-Code and GitOps methodologies
- Improve deployment velocity and reliability
- Support engineering teams with self-service platform capabilities
- GitHub Actions, GitLab CI/CD, Jenkins, Azure DevOps, Terraform, Docker, Kubernetes
- Support API integrations and operational workflows
- Establish Site Reliability Engineering (SRE) practices
- Define platform SLAs, SLOs, and operational metrics
- Lead platform monitoring, observability, and incident management
- Drive root cause analysis and preventive actions
Skills
- Data Bricks
- EBT
- Airflow
- Asset Management experience
- Deep expertise in Databricks
- Expertise in dbt
- Expertise in AWS Cloud
- Expertise in Terraform
- Expertise in Python
- Expertise in DevOps
- Expertise in Platform Engineering
- Expertise in Data Operations
- Strong experience supporting investment management organizations
- Experience in designing, implementing, and managing enterprise data platforms
- Ability to define platform engineering strategy aligned with EDO and enterprise architecture standards
- Experience in establishing reusable platform frameworks, accelerators, and engineering best practices
- Experience in driving cloud-native modernization initiatives across data and analytics ecosystems
- Experience in owning platform lifecycle management, scalability, availability, and reliability
- Experience in leading enterprise-wide Databricks platform architecture and administration
- Experience in designing and implementing Lakehouse architecture leveraging Delta Lake
- Experience in managing Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, MLflow, Structured Streaming, Cluster Policies, Security and Access Controls
- Experience in optimizing platform performance and cost efficiency
- Experience in building governance controls across development, test, and production environments
- Experience in leading cloud engineering activities using AWS services: S3, ECS, EKS, Lambda, Glue, Cloudwatch
- Experience in Infrastructure as a Code (IaC)
- Experience in building reusable Terraform modules
- Experience in automating environment provisioning
- Experience in implementing infrastructure standardization and governance
- Experience in maintaining infrastructure version control
- Experience in ensuring compliance through Infrastructure-as-Code practices
- Experience in designing and implementing CI/CD pipelines
- Experience in automating deployment and release management processes
- Experience in enabling Infrastructure-as-Code and GitOps methodologies
- Experience in improving deployment velocity and reliability
- Experience in supporting engineering teams with self-service platform capabilities
- Experience with GitHub Actions, GitLab CI/CD, Jenkins, Azure DevOps, Terraform, Docker, Kubernetes
- Experience in supporting API integrations and operational workflows
- Experience in establishing Site Reliability Engineering (SRE) practices
- Experience in defining platform SLAs, SLOs, and operational metrics
- Experience in leading platform monitoring, observability, and incident management
- Experience in driving root cause analysis and preventive actions
Company Overview
Company H1B Sponsorship