[Remote] Databricks Engineer
Note: The job is a remote job and is open to candidates in USA. Blu Omega is seeking a Databricks Engineer to support a federal program focused on data analytics and mission-critical government operations. The role designs, builds, operates, and optimizes scalable Databricks data solutions, including data pipelines, cloud-based platforms, CI/CD integrations, data quality frameworks, and production workloads.
Responsibilities
- Design, develop, and maintain scalable Databricks data pipelines using PySpark, Spark SQL, Delta Lake, and Unity Catalog to deliver high-quality, mission-critical data assets
- Build and maintain automated ingestion, transformation, and processing workflows supporting analytics, reporting, and operational decision-making
- Operate and optimize Databricks clusters, jobs, and workflows ensuring reliability, performance, and cost efficiency
- Integrate Databricks with enterprise CI/CD pipelines and SDLC tooling for automated deployments, source control, and testing
- Implement data quality, validation, and observability frameworks to enhance data accuracy and trustworthiness
- Monitor and troubleshoot data pipelines, platform performance, and production workloads to identify and resolve issues proactively
- Collaborate with stakeholders and cross-functional teams to translate operational requirements into technical solutions
- Develop and maintain technical documentation related to data pipelines, architecture, and operational processes
Skills
- 3+ years of experience in data engineering, analytics, or related technical roles
- 2+ years of hands-on experience with the Databricks platform
- Strong experience developing data solutions using PySpark and Spark SQL
- Experience working with Databricks clusters, jobs, workflows, Delta Lake, and Unity Catalog in production environments
- Experience designing and maintaining automated data pipelines and transformation workflows
- Experience integrating Databricks with CI/CD pipelines and enterprise SDLC tooling
- Working knowledge of Python for data engineering, automation, and ML workflows
- Knowledge of cloud platforms such as AWS, Microsoft Azure, or GCP and associated data services
- Understanding of data quality, validation, monitoring, and production support practices
- Strong analytical and troubleshooting skills
- Effective collaboration skills with technical and non-technical teams
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination
- Bachelor's degree in computer science, data science, engineering, or related field
- Experience supporting federal government or regulated industries
- Experience developing or operationalizing machine learning solutions within Databricks
- Experience with MLflow for machine learning lifecycle management
- Knowledge of streaming frameworks, real-time data processing, and observability tooling
- Experience implementing data governance, lineage, and access control practices
- Databricks certification or cloud platform certification from AWS, Azure, or GCP
Benefits
- Medical, Dental, and Vision coverage through national providers
- 401(k) with company match (eligible after 6 months; vesting applies)
- Company-paid Life and AD&D insurance with additional voluntary options
- Short-term and long-term disability options
- Employee Assistance Program (EAP) with 24/7 confidential support and mental health resources
- Telehealth and virtual care options
- Pet insurance, legal services, and identity theft protection options
- Paid Time Off (PTO) for eligible employees
- Paid federal holidays for salaried employees
- Access to wellness programs, discounts, and lifestyle benefits
- Benefits availability may vary based on role and employment status
Company Overview