[Remote] Tech Lead – Data Engineering (Databricks) & AI
Note: The job is a remote job and is open to candidates in USA. EazyML is a Gartner-recognized company specializing in Responsible AI and enterprise AI adoption. The company is seeking a Tech Lead to design, develop, and deliver scalable Databricks-based data engineering and AI/ML solutions while providing hands-on technical leadership, mentoring engineers, and engaging with stakeholders.
Responsibilities
- Lead the technical design and architecture of large-scale data engineering and AI/ML solutions on Databricks (Lakehouse Platform)
- Provide hands-on technical leadership across the full data lifecycle: ingestion, transformation, storage, and consumption using Spark, Delta Lake, and Databricks workflows
- Design and implement scalable ETL/ELT pipelines, ensuring performance, reliability, and cost optimization
- Guide the team in building and deploying AI/ML models and GenAI/LLM-based solutions, integrating them into production data pipelines
- Collaborate with data scientists, ML engineers, and business stakeholders to translate requirements into robust technical solutions
- Establish and enforce best practices for data governance, security, CI/CD, and DataOps/MLOps within Databricks environments
- Mentor and coach a team of data engineers, conducting code reviews and driving engineering excellence
- Own technical decision-making, including platform architecture, tool selection, and performance tuning
- Act as the primary technical point of contact for client/stakeholder discussions, ensuring alignment between business goals and technical execution
- Stay current with evolving Databricks features, Delta Lake updates, and AI/GenAI trends, driving adoption of relevant innovations
Skills
- 8+ years of overall experience in data engineering, with 3+ years in a technical leadership or lead role
- Strong hands-on experience with Databricks, including:
- Spark (PySpark/Scala), Delta Lake, Unity Catalog
- Databricks Workflows/Jobs, Cluster management & optimization
- Databricks SQL and performance tuning
- Solid experience in Data Engineering, including:
- Building and optimizing ETL/ELT pipelines at scale
- Data modeling (batch and streaming)
- Working with cloud data platforms (Azure/AWS/GCP – specify as needed)
- Experience with orchestration tools (e.g., Airflow, ADF, Databricks Workflows)
- Practical experience in AI/ML, including:
- Building, training, and deploying ML models
- MLOps practices for model lifecycle management
- Strong programming skills in Python and SQL; familiarity with Scala is a plus
- Experience with CI/CD pipelines and version control (Git-based workflows)
- Solid understanding of data security, governance, and compliance best practices
- Excellent communication and stakeholder management skills, with the ability to work directly with onshore clients/business teams
- Proven experience leading and mentoring technical teams
- Strong ownership mindset with the ability to balance hands-on delivery and team leadership
- Excellent problem-solving and analytical thinking
- Strong communication skills to bridge technical and business conversations
- Ability to work independently in a fast-paced, onshore client-facing environment
- This is a fully remote role with the flexibility to work from anywhere in USA, must have authorization to work in US
- Familiarity with GenAI/LLM frameworks (e.g., LangChain, RAG pipelines, vector databases) is a strong plus
- Experience in the Life Sciences domain (pharma, biotech, clinical, or healthcare data) is preferred
- Databricks certification(s) (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Machine Learning Professional)
- Cloud certification (Azure/AWS/GCP)
- Experience in Agile/Scrum delivery environments
- Prior experience working in client-facing consulting or professional services roles
Benefits
- This is a fully remote role with the flexibility to work from anywhere in USA
Company Overview