[Remote] Databricks Data Engineer
Note: The job is a remote job and is open to candidates in USA. Peraton is a next-generation national security company that drives missions of consequence spanning the globe. The Databricks Data Engineer will be responsible for building and configuring a Government-owned Databricks workspace, designing data pipelines, and implementing data governance solutions.
Responsibilities
- Build out and configure a dedicated Government workspace within the existing Databricks environment
- Design and implement data ingestion pipelines from core agency business systems including financial, HR, CRM, and ITSM systems
- Leverage native/built-in connectors where source systems support them; design custom integration approaches for legacy systems
- Normalize and prepare ingested data within Databricks for consumption by downstream visualization/reporting tools
- Implement Databricks Unity Catalog for centralized data governance, metadata management, active auditing, and end-to-end lineage tracking
- Review current configuration, assess security controls for CUI/PII/PHI/financial data, and implement improvements
- Develop comprehensive 'as-built' documentation including physical/logical architecture diagrams, automated data dictionaries, and SOPs
- Document data sources, integration methods, and data lake architecture decisions to support sustainment
Skills
- 5 years work experience with BS/BA; 3 years with MS/MA
- US Citizenship
- Active DoD Secret clearance
- 2 years working on the Databricks platform
- Strong proficiency in PySpark, Spark SQL, and Python for large-scale data processing and pipeline development
- Hands-on experience with Unity Catalog administration, including metastore management, access policies, and data lineage
- Experience with pipeline orchestration tools (Airflow, Databricks Workflows)
- Experience with Delta Lake, schema evolution, time travel, and optimization techniques
- Experience integrating heterogeneous enterprise systems, including legacy/custom integrations
- Familiarity with cloud platforms (Azure/AWS) and infrastructure-as-code practices
- Understanding of data governance principles, compliance frameworks, and CUI/PII handling practices
- Familiarity with financial, HR, CRM, or ITSM data structures
- Git/CI-CD pipeline experience
- DoD 8570 certification
- Databricks Certified Data Engineer Associate or Professional certification
Company Overview