[Remote] Data Platform Engineer

Note: The job is a remote job and is open to candidates in USA. Greystar is a global real estate platform offering property management, investment management, development, and construction services. The Data Platform Engineer will own the reliability, governance, cost efficiency, and operations of the company’s Databricks-native data infrastructure on Microsoft Azure. The role focuses on platform administration, CI/CD, identity and access governance, observability, self-healing capabilities, disaster recovery, and multi-platform data services support.

Responsibilities

• Own Databricks workspace administration: upgrades, cluster policies, compute configurations, and Delta table lifecycle management
• Administer Unity Catalog: access controls, service principal (SP), group, and user provisioning, governance policies, and data lineage
• Tune Spark jobs for performance, reliability, and cost — profiling bottlenecks, optimizing partitioning, managing Z-ordering, and controlling compute spend
• Architect and enforce DMP platform standards: naming conventions, schema evolution policies, SLA tiers, and medallion layer contracts
• Leverage Databricks Mosaic AI and Genie to build AI-native DataOps capabilities including intelligent pipeline monitoring and anomaly detection
• Operate the full Azure data services stack: ADLS Gen2, Azure Data Factory (ADF), Azure Monitor, Log Analytics, Key Vault, and Event Hub
• Collaborate with Azure infrastructure and cloud engineering teams on networking, identity, security, and resource provisioning
• Drive cost governance through Azure Cost Management, Databricks DBU optimization, and storage lifecycle policies
• Provide platform support for various systems and platforms including access management, storage optimization, and compliance
• Support additional platforms including ADF, Synapse Analytics, Cosmos DB, and Azure SQL Server
• Manage storage allocation, cluster optimization, and resource provisioning across all supported platforms
• Support compliance, audit readiness, and Key Vault secret rotation and management
• Own the full deployment pipeline for DMP data workflows — promoting changes from development through staging to production
• Build and maintain CI/CD workflows using GitHub Enterprise: branch strategies, PR automation, environment-specific configuration management, and release gating
• Enforce deployment standards: automated testing gates, rollback procedures, change documentation, and environment parity controls
• Use Linear for sprint planning, release tracking, and issue management across deployment cycles
• Manage Azure Active Directory (AAD) service principals, RBAC assignments, and Unity Catalog permission policies
• Partner with the Data Governance team to enforce data contracts, ownership standards, and quality SLAs within Unity Catalog
• Maintain data lineage, metadata management, and governance framework documentation
• Ensure platform configurations meet security, compliance, and audit requirements
• Own workspace configuration parity across Dev, Staging, and Production Databricks environments — cluster policies, Unity Catalog hierarchy, access controls, and compute configurations
• Define and enforce environment promotion hygiene: configuration drift detection, environment-specific secret management, and pre-production validation gates
• Maintain infrastructure-as-code (IaC) scripts and configuration templates to ensure reproducible, auditable environment provisioning across all tiers
• Produce thorough technical documentation: runbooks, deployment playbooks, incident post-mortems, ADRs, and platform specs
• Partner with analytics engineers, data governance, and product stakeholders on platform design and capacity planning
• Participate in on-call rotation and support SLA commitments for business-critical DMP data domains
• Build and maintain AI-powered platform observability infrastructure — using LLMs and ML models to detect pipeline drift, classify anomalies, predict SLA risk, and generate automated incident summaries at the platform layer
• Architect and deploy self-healing pipeline capabilities — automated retry logic, circuit breakers, and remediation triggers — as production-grade platform features, not experiments
• Integrate Databricks Mosaic AI and Genie into operational platform workflows for intelligent cluster management, anomaly detection, and natural language platform diagnostics
• Contribute to Greystar’s 18-month agentic AI roadmap, leading near-term delivery of platform-level self-healing and proactive alerting capabilities
• Own DMP pipeline architecture patterns from an infrastructure standpoint — Delta Live Tables (DLT) design standards, schema evolution enforcement, and medallion layer contracts
• Define and enforce platform-wide naming conventions, partitioning strategies, Z-ordering policies, and Delta table optimization schedules (VACUUM, OPTIMIZE)
• Partner with analytics engineers to ensure pipeline designs conform to platform infrastructure standards, SLA tiers, and compute budget constraints
• Design and maintain DR strategies for the DMP platform — including Delta table backup policies, ADLS Gen2 geo-redundancy configuration, and Databricks workspace recovery runbooks
• Define RTO and RPO targets for business-critical DMP data domains; test and validate recovery procedures on a recurring basis
• Maintain business continuity plans aligned with Azure region failover capabilities and Greystar’s enterprise SLA commitments
• Participate in on-call rotation coverage across assigned shift window
• Manage ServiceNow / Asana ticket queues per team SLA commitments
• Track and close Linear / DevOps work items as part of sprint cycles
• Contribute to team knowledge base, runbook library, and post-incident documentation

Skills

• 5+ years of platform engineering, infrastructure engineering, or data operations experience in a production environment
• Expert-level Databricks administration: Unity Catalog, cluster policies, workspace management, compute configuration, and Delta Lake internals
• Strong command of the Azure data services ecosystem: ADLS Gen2, ADF, Azure Monitor, Log Analytics, Key Vault, Cost Management, and Event Hub
• Identity and access management: Azure Active Directory, service principals, RBAC, and Unity Catalog governance policies
• Hands-on experience with ADF pipeline design and orchestration at scale
• Spark performance tuning: partitioning, Z-ordering, compute spend optimization, and bottleneck profiling
• Proven CI/CD experience using GitHub Enterprise: environment promotion, rollback, and release management for data pipelines
• Python and/or SQL for automation scripts, compliance validation, and platform tooling
• Knowledge of medallion / lakehouse architecture patterns and multi-environment deployment discipline
• Terraform / Infrastructure-as-Code (IaC): provisioning and managing Databricks workspaces, Azure resources, Unity Catalog configurations, and environment-specific infrastructure through version-controlled IaC
• Azure DevOps Pipelines: build and release pipeline configuration, YAML pipeline authoring, and integration with GitHub Enterprise for end-to-end deployment automation
• Azure networking: VNet configuration, private endpoints for Databricks and ADLS Gen2, NSG rules, firewall policies, and DNS resolution for secure data platform connectivity
• Azure Monitor & Log Analytics: KQL query authoring, alert rule configuration, diagnostic settings, workbook creation, and cluster event monitoring for proactive platform health management
• Delta Lake internals: VACUUM, OPTIMIZE, ZORDER, time travel, table statistics, and transaction log management for production-grade Delta table lifecycle governance
• Experience with Synapse Analytics, Cosmos DB, or Azure SQL Server in production data environments
• Familiarity with Databricks Mosaic AI, Genie, or AI-native platform observability capabilities
• Background in data lineage tooling, metadata management, and Unity Catalog governance frameworks
• Experience with ERP system integrations: Yardi, Entrata, or RealPage in multi-tenant environments
• Background in legacy BI migration or platform modernization programs
• Linear for engineering sprint and release management

Benefits

• Competitive Medical, Dental, Vision, and Disability & Life insurance benefits. Low (free basic) employee Medical costs for employee-only coverage; costs discounted after 3 and 5 years of service.
• Generous Paid Time off. All new hires start with 15 days of vacation, 4 personal days, 10 sick days, and 11 paid holidays. Plus your birthday off after 1 year of service! Additional vacation accrued with tenure.
• For onsite team members, onsite housing discount at Greystar-managed communities are available subject to discount and unit availability.
• 6-Week Paid Sabbatical after 10 years of service (and every 5 years thereafter).
• 401(k) with Company Match up to 6% of pay after 6 months of service.
• Paid Parental Leave and lifetime Fertility Benefit reimbursement up to $10,000 (includes adoption or surrogacy).
• Employee Assistance Program.
• Critical Illness, Accident, Hospital Indemnity, Pet Insurance and Legal Plans.
• Charitable giving program and benefits.
• This position may be performed remotely anywhere within the United States except the state of Alaska.

Company Overview

Company H1B Sponsorship