Data Engineer
Job Description: Role: Senior Data Engineer Location: Remote Duration: Full Time
Job Description
Seeking a Senior Data Engineer with strong hands-on experience in Databricks, Spark, and AWS to build scalable, secure data pipelines and lakehouse solutions supporting analytics, AI/ML, and healthcare data.
Key Responsibilities
Design and build data pipelines using Databricks, Delta Lake, Spark, Unity Catalog, and Workflows. Develop batch and real-time pipelines using AWS S3, Glue, Redshift, Athena, Lambda, Step Functions, and Kinesis. Build and optimize lakehouse architecture, data models, and ETL/ELT processes. Support ML/AI initiatives including feature stores, LLM/RAG data pipelines, and data integration. Implement data security, governance, and HIPAA-compliant data handling, including access controls and auditing. Partner with Data Science, Engineering, Product, and business teams. Review code, establish engineering best practices, and mentor junior engineers. Requirements 8+ years of Data Engineering experience. 4+ years working with cloud-based data lake/lakehouse architectures. Deep hands-on experience with Databricks, Apache Spark/PySpark, Delta Lake, Unity Catalog, and Workflows. Strong AWS experience with S3 and data/analytics services. Strong Python and SQL skills. Experience with Airflow/dbt, Kafka/Kinesis, CI/CD, Docker/Kubernetes, and data quality practices. Healthcare or regulated-data experience with HIPAA required. Experience supporting ML/AI data pipelines and feature stores. Strong communication and technical leadership skills. Thanks, Suresh- Sr. Technical Recruiter