[Remote] Distinguished Engineer, Data Platform

Note: The job is a remote job and is open to candidates in USA. CloudZero is a SaaS platform focused on cloud cost management and FinOps, aiming to help organizations make informed business decisions through real-time data organization. As a Distinguished Engineer on the Data Engineering team, you will tackle complex infrastructure problems, lead the design of a next-generation data platform, and drive the evolution of the company's data architecture and query engine.

Responsibilities

• Define the Data Platform Architecture
• Lead end-to-end technical design for CloudZero's next-generation data platform, from event ingestion and stream processing through hot/cold storage and the query layer to the API surface
• Document architectural decisions, tradeoffs, and migration strategies with the rigor of an RFC-driven process
• Shape and drive every layer of the new architecture: event ingestion, stream processing and enrichment, real-time serving, analytical storage, query layer, and API
• Design and deliver CloudZero's real-time data pipeline from ingestion through enrichment to serving
• Establish SLOs for throughput, latency, and correctness, and build the operational playbooks that make this system trustworthy enough to replace the batch pipelines our entire product currently depends on
• Tackle real-time streaming at scale across thousands of customers simultaneously, with fault tolerance, backpressure awareness, and correctness as non-negotiables
• Redesign CloudZero's dimensional cost model to support high-cardinality, multi-dimensional cost attribution without runaway materialization costs
• Drive incremental, delta-based materialization strategies using modern open table formats, dramatically reducing expensive full-rebuild jobs and unlocking millions in annual infrastructure savings
• Assess CloudZero's current query infrastructure, drive in-flight migrations to completion, and lead the evolution of the query engine layer going forward
• Own performance optimization across partition pruning, predicate pushdown, and query planning, and set the vision for how the query layer grows as data volumes scale 10x
• Evolve CloudZero's proprietary cost attribution engine from a batch-oriented model to one that assigns complex cost dimensions by team, feature, and customer within seconds of resource usage
• Rethink enrichment, data lineage, and correctness guarantees in a streaming context
• Partner with product, infrastructure, and analytics engineering to define a multi-year data platform roadmap
• Build consensus across engineering leadership on foundational investments including table formats, streaming frameworks, query engines, and schema management
• Participate in architecture reviews, contribute to design patterns and best practices, and mentor senior and staff engineers through code review, pairing, and structured feedback
• Make everyone around you better, not by directing, but by raising the collective craft

Skills

• 10+ years in data engineering with a clear trajectory toward principal or staff-level architecture
• Built and operated large-scale data platforms serving tens of millions of events per day in production
• Deep experience with streaming systems such as Kafka, Kinesis, Flink, or Spark Streaming at real production throughput
• Strong hands-on fluency with modern open table formats including Apache Iceberg, Delta Lake, and Hudi, including compaction, partitioning strategy, and time-travel queries
• Designed hot/cold storage architectures with explicit latency SLOs per tier
• Proven ability to drive a data platform end to end, not just a single layer
• Expert in dimensional data modeling including fact/dimension schema design, slowly changing dimensions, and cardinality management
• Deep understanding of the materialization tradeoff space: full vs. incremental, push vs. pull, pre-aggregate vs. query-time
• Experience with cost attribution, showback/chargeback, or multi-tenant data partitioning patterns
• Strong SQL and query optimization background across predicate pushdown, partition pruning, and cost-based query planning
• Hands-on with distributed query engines such as Trino, Presto, Spark SQL, or DuckDB including configuration, optimization, and production operations
• Understands catalog and metadata management and how it couples to query engines
• Comfortable with cloud data warehouses such as Snowflake, BigQuery, and Redshift and how they integrate with open table formats
• Experience driving query engine migrations while maintaining production SLAs
• Track record as a technical anchor for a data platform or data engineering team
• Writes clear ADRs, RFCs, and technical design docs that bring engineers along
• Can drive multi-month, multi-team technical initiatives from inception to production without heavy process overhead
• Communicates complex tradeoffs to non-technical stakeholders including product and business leadership
• Comfortable in a high-autonomy environment: builds consensus, influences through expertise, and helps teams move forward
• FinOps or cloud cost domain experience
• Multi-cloud data ingestion across AWS, Azure, and GCP
• Apache Flink at production scale
• Lakehouse architecture patterns
• Real-time feature engineering for ML
• Data mesh or domain-oriented design patterns
• Prior startup or high-growth SaaS experience
• Open source contributions to the data ecosystem

Benefits

• Offers Equity
• Offers Bonus
• Remote

Company Overview