[Remote] AWS Data Engineer
Note: The job is a remote job and is open to candidates in USA. LHH is seeking a Senior Data Engineer to support Platform Ventures’ critical data systems and services. The role focuses on designing scalable AWS data architectures, building batch and streaming pipelines, implementing governance and automation, and collaborating with data consumers, analysts, platform engineers, vendors, and executives.
Responsibilities
- The candidate will design and develop data pipelines, data marts, and schemas to support our data warehousing environments and analytical requirements
- Design and implement scalable data Lakehouse architectures using AWS S3, Delta Lake, and Redshift
- Build and maintain high-performance data pipelines to ingest, transform, and curate large datasets
- Define best practices for data modeling, partitioning, and storage optimization in a Lakehouse/medallion environment
- Develop and maintain data cataloging and metadata management solutions
- Develop ETL/ELT pipelines using Python, PySpark, and AWS Glue
- Build robust batch and streaming data pipelines
- Implement transformation workflows using DBT
- Optimize performance for Spark workloads and Redshift queries
- Design and maintain workflow orchestration using Apache Airflow
- Monitor pipeline performance and implement alerting and failure recovery
- Implement data validation, lineage, and monitoring frameworks
- Maintain data documentation and data catalog integration
- Ensure data reliability, consistency, and security across platforms
- Implement CI/CD pipelines for data workflows
- Automate testing, deployment, and monitoring of data pipelines
- Use infrastructure-as-code where possible to maintain reproducible environments
- Work closely with data consumers, analysts, and platform engineers
- Support self-service analytics through well-structured datasets and semantic models
Skills
- MUST HAVE CURRENT AWS experience
- United States
- Fully remote
- Strong Communication with vendors, customers, C-Suite executives, and co workers is imperative
- • 7+ years of data warehousing / OLAP experience
- • 7+ years multidimensional database architecture, data modeling, and design experience
- • 5+ years of SQL Experience
- • 3+ years of Python Experience
- • 2+ years of Redshift Experience
- • 2+ years of NoSQL Experience
- • Team and vendor partner management
- • Clear understanding of principles and techniques of Web development
- • Experience with AWS Glue, Lambda, Step Functions, and Airflow
- • Experience with data ingression from APIs
- • Dimensional database architecture experience
- • Formal systems life cycle methodology experience
- • Experience implementing CI/CD pipelines for data workflows
- • Experience working with data catalogs and metadata management
- • AWS Data Warehouse operations experience
- • Tableau, Quicksight, Looker, big data and Realtime BI experience
- • Experience with data streaming
Benefits
- 401K
- Medical and Dental
- Annual bonus
- Fully remote
Company Overview