[Remote] Databricks Data Engineer
Note: The job is a remote job and is open to candidates in USA. Peraton is a national security and enterprise IT company that supports government agencies and defense missions. The Databricks Data Engineer will build and configure a Government-owned Databricks workspace, develop data ingestion and integration pipelines, implement governance and machine learning capabilities, and create documentation for long-term sustainment.
Responsibilities
- Build out and configure a dedicated Government workspace within the existing Databricks environment
- Design and implement data ingestion pipelines from core agency business systems including financial, HR, CRM, and ITSM systems
- Leverage native/built-in connectors where source systems support them; design custom integration approaches for legacy systems
- Normalize and prepare ingested data within Databricks for consumption by downstream visualization/reporting tools
- Implement Databricks Unity Catalog for centralized data governance, metadata management, active auditing, and end-to-end lineage tracking
- Review current configuration, assess security controls for CUI/PII/PHI/financial data, and implement improvements
- Develop comprehensive "as-built" documentation including physical/logical architecture diagrams, automated data dictionaries, and SOPs
- Document data sources, integration methods, and data lake architecture decisions to support sustainment
Skills
- Min 8 years with BS/BA, Min 6 years with MS/MA, Min 3 years with PhD
- US Citizenship
- Active DoD Secret clearance
- 2 years working on the Databricks platform
- Strong proficiency in PySpark, Spark SQL, and Python for large-scale data processing and pipeline development
- Hands-on experience with Unity Catalog administration, including metastore management, access policies, and data lineage
- Experience with pipeline orchestration tools (Airflow, Databricks Workflows)
- Experience with Delta Lake, schema evolution, time travel, and optimization techniques
- Experience integrating heterogeneous enterprise systems, including legacy/custom integrations
- Familiarity with cloud platforms (Azure/AWS) and infrastructure-as-code practices
- Understanding of data governance principles, compliance frameworks, and CUI/PII handling practices
- Familiarity with financial, HR, CRM, or ITSM data structures
- Git/CI-CD pipeline experience
- DoD 8570 certification
- Databricks Certified Data Engineer Associate or Professional certification
Benefits
- 100% Remote
- Employees may be eligible for overtime, shift differential, and a discretionary bonus in addition to base pay.
Company Overview