[Remote] Databricks Data Engineer - Clearance Required
Note: The job is a remote job and is open to candidates in USA. The Squires Group is seeking a Senior Databricks Data Engineer for a federal technology modernization program. The role builds and supports Databricks data pipelines and migrates data and business logic from legacy systems to cloud-based environments.
Responsibilities
- Build, operate, and maintain ingestion pipelines within Databricks
- Develop and support Bronze, Silver, and Gold data pipelines using Medallion Architecture
- Migrate data and business logic from legacy systems into modern cloud-based environments
- Gather and analyze technical and business requirements
- Design and develop batch, incremental, and streaming ingestion pipelines
- Monitor data jobs and troubleshoot pipeline and production issues
- Provide post-go-live production support and break/fix assistance
- Document technical processes, data workflows, and procedures
- Collaborate with technical teams and stakeholders in an Agile environment
Skills
- An active DoD Secret clearance and U.S. citizenship are required for this position
- Strong experience in data engineering, ETL/ELT development, or data warehousing
- Strong PySpark and/or Spark SQL skills with solid SQL fundamentals
- Experience with Databricks and Medallion Architecture, including Bronze, Silver, and Gold layering
- Experience with Delta Lake, including ACID transactions, schema evolution, time travel, and MERGE/upsert patterns
- Experience designing batch and incremental/streaming ingestion pipelines
- Familiarity with Agile processes
- Excellent written and verbal communication skills
- Strong interpersonal, organizational, and prioritization skills
- Databricks certification or additional hands-on Databricks experience preferred
- DoD, Navy, or other federal experience preferred
- AWS GovCloud experience preferred
- Experience migrating code from non-accredited to accredited AWS environments is a plus
Benefits
- Depending on compensation type, benefits may include paid time off (PTO), medical, dental, and vision coverage, life insurance, long-term disability insurance, a 401(k) plan, and additional optional benefits.
- Fully remote position.
- Multi-year projects.
- Support to help you meet your career goals.
Company Overview