[Remote] Lead Data DevOps Engineer
Note: The job is a remote job and is open to candidates in USA. EPAM Systems is seeking a Lead Data DevOps Engineer to help build an Enterprise AI Gateway from scratch. The role focuses on automating onboarding, infrastructure, governance, monitoring, and cost tracking for large language model usage while ensuring reliable production operations and influencing platform architecture.
Responsibilities
- Design and establish the core structure of the Enterprise AI Gateway from the very beginning
- Build automation for onboarding teams, agents, and MCP servers, covering the setup of keys, permissions, limits, and guardrails
- Develop and sustain infrastructure as code to enable scalable, self-service capabilities across the platform
- Implement monitoring, logging, and cost-tracking tools to maintain clear oversight of AI usage
- Create guardrails and governance frameworks that regulate which teams and models can reach specific resources
- Ensure production systems supporting the gateway remain reliable, available, and performant
- Engage directly with platform users, answering questions and helping troubleshoot problems as they come up
- Continuously enhance automation to cut down on manual effort and dependence on ticket-driven processes
- Review and integrate new GenAI and agentic AI patterns, frameworks, and protocols into the platform
- Help drive important architectural and design choices as the platform grows beyond its early stages
Skills
- A minimum of 5 years of relevant experience
- At least one year of experience leading and managing teams
- Strong Python skills applied to automation, extensions, and integrations
- Solid SRE experience, with a genuine track record of maintaining stable production systems
- Skilled with Git for version control
- Background working with Google Cloud Platform
- Familiarity with LLMOps practices
- Experience applying Terraform and Helm for infrastructure automation
- Working knowledge of GenAI/Agentic AI concepts, including related patterns, frameworks, and protocols
- Excellent communication skills, with the ability to explain technical topics clearly, given daily interaction with platform users to answer questions and resolve issues
- Excellent English proficiency (B2 level or higher)
- Practical experience with Google Vertex AI, particularly endpoints for model serving and Model Armor
- GCP experience covering services such as BigQuery, Cloud Run, and IAM
- Experience developing AI agents, for example through Google's Agent Development Kit (ADK)
- Experience with AWS Bedrock
- Broad Kubernetes experience, ideally involving GKE
- Familiarity with Groovy
- Experience with CI/CD pipelines built on Jenkins
- Production experience with an AI gateway, such as LiteLLM or EPAM DIAL, seen as highly valuable
Benefits
- International projects with top brands
- Work with global teams of highly skilled, diverse peers
- Healthcare benefits
- Employee financial programs
- Paid time off and sick leave
- Upskilling, reskilling and certification courses
- Unlimited access to the LinkedIn Learning library and 22,000+ courses
- Global career opportunities
- Volunteer and community involvement opportunities
- EPAM Employee Groups
- Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn
Company Overview
Company H1B Sponsorship