[Remote] Data Engineer, Databricks
Note: The job is a remote job and is open to candidates in USA. KeyLogic is seeking Databricks Engineers to support the Department of Energy (DOE). The Data Engineer will design, build, and maintain scalable data pipelines, optimize data workflows, and implement infrastructure automation using Databricks and Terraform. The role also involves collaborating with cross-functional teams, monitoring data systems, and ensuring data quality and reliability.
Responsibilities
- Design, build, and maintain scalable data pipelines using Databricks
- Develop and implement infrastructure as code solutions with Terraform
- Collaborate with cross-functional teams to understand data requirements and deliver robust solutions
- Optimize data workflows for performance, reliability, and scalability
- Monitor, troubleshoot, and resolve issues related to data processing and infrastructure
- Ensure data quality and integrity throughout the data lifecycle
- Automate repetitive tasks to improve efficiency and reduce manual intervention
Skills
- U.S. Citizenship
- 2+ years of engineering experience on Databricks and large-scale big data projects (Delta Lake, DLT, Apache Spark, unity Catalog)
- Bachelor's Degree
- 4+ years of experience in Python, Java, or equivalent. Comfortable with evaluation tooling, logging, monitoring, and observability
- Experience with government contracting
Benefits
- Remote work arrangement
- Full-time employment
Company Overview