[Remote] DataOps Engineer

Note: The job is a remote job and is open to candidates in USA. Greystar is a global real estate platform offering property management, investment management, development, and construction services. The DataOps Engineer will own the reliability, scalability, observability, and deployment lifecycle of the Databricks-native enterprise data platform on Microsoft Azure, while developing AI-driven monitoring and self-healing pipeline capabilities. The role also covers Azure infrastructure integration, data quality and governance, CI/CD, platform optimization, documentation, and on-call support.

Responsibilities

• Implement AI-powered observability — using LLMs and ML models to detect pipeline drift, classify anomalies, predict SLA risk, and generate automated incident summaries
• Build agentic monitoring workflows that proactively surface data quality degradation, pipeline dropout, schema drift, and volume anomalies across all DMP layers
• Integrate AI tooling (Databricks Mosaic AI, Genie, OpenAI APIs, or equivalent) into operational DataOps processes — not as experiments, but as production-grade capabilities
• Develop and maintain AI-assisted root cause analysis tooling to reduce MTTR on pipeline failures, with structured learnings fed back into the platform
• Contribute to Greystar’s 18-month agentic AI roadmap, leading near-term delivery of self-healing pipeline capabilities
• Operate the full Azure data services stack supporting DMP: ADLS Gen2, Azure Data Factory (ADF), Azure Monitor, Log Analytics, Key Vault, and Event Hub
• Design and maintain ADF pipelines for source system ingestion, including orchestration patterns for multi-tenant ERP environments (Yardi, Entrata, RealPage)
• Collaborate with Azure infrastructure and cloud engineering teams on networking, identity, security, and resource provisioning
• Drive cost governance through Azure Cost Management, Databricks DBU optimization, and storage lifecycle policies
• Own the design, build, and optimization of data pipelines on Databricks using Delta Live Tables (DLT), PySpark, Workflows, and Jobs across the full DMP medallion stack
• Administer and govern the Databricks workspace: Unity Catalog, cluster policies, access controls, compute configurations, and Delta table lifecycle management
• Tune Spark jobs for performance, reliability, and cost — profiling bottlenecks, optimizing partitioning, managing Z-ordering, and controlling compute spend
• Leverage Databricks Mosaic AI and Genie to build AI-native DataOps capabilities including intelligent pipeline monitoring, anomaly detection, and natural language data access
• Architect and enforce DMP platform standards: naming conventions, schema evolution policies, SLA tiers, and medallion layer contracts
• Own the full deployment pipeline for DMP data workflows — promoting changes from development through staging to production with rigor and minimal disruption
• Build and maintain CI/CD workflows using GitHub Enterprise, including branch strategies, pull request automation, environment-specific configuration management, and release gating
• Use Linear for sprint planning, release tracking, and issue management across deployment cycles; coordinate engineering work items with cross-functional stakeholders
• Enforce deployment standards: automated testing gates, rollback procedures, change documentation, and environment parity controls
• Partner with the analytics engineering and integration teams to align deployment cadences across the DMP stack
• Instrument DQ checks across Bronze, Silver, and Gold layers covering completeness, consistency, accuracy, uniqueness, and referential integrity
• Partner with Brett Finley’s Data Governance team to enforce data contracts, ownership standards, and quality SLAs within Unity Catalog
• Build feedback loops between DQ scoring, pipeline observability, and upstream source owners to drive systemic data reliability improvements
• Partner with analytics engineers, data governance, and product stakeholders to align pipeline and platform design with business requirements
• Produce thorough technical documentation — runbooks, deployment playbooks, incident post-mortems, ADRs, and platform specs
• Participate in on-call rotation and support SLA commitments for business-critical DMP data domains

Skills

• 7+ years of DataOps, data engineering, or platform engineering experience in a production environment
• Expert-level hands-on experience with Databricks: Delta Live Tables, Jobs/Workflows, Unity Catalog, Spark performance tuning, and Delta Lake internals
• Strong command of the Azure data services ecosystem: ADF, ADLS Gen2, Azure Monitor, Log Analytics, Key Vault, and related services
• Demonstrated, production use of AI tools in DataOps or data observability workflows — LLM-assisted diagnostics, intelligent alerting, agentic monitoring, or equivalent
• Proven CI/CD experience using GitHub Enterprise — branch strategies, PR automation, environment promotion, and release management for data pipelines
• Solid Python and/or Scala skills for pipeline development; SQL fluency for Gold layer transformation and DQ validation
• Hands-on experience with ADF pipeline design and orchestration at scale
• Experience with medallion / lakehouse architecture patterns and multi-environment deployment discipline
• Strong collaborative skills across engineering, governance, and business stakeholder teams
• Experience with Linear for engineering sprint management and release tracking
• Familiarity with Databricks Mosaic AI, Genie, or other AI-native Databricks capabilities
• Exposure to agentic AI frameworks or MCP (Model Context Protocol) server integrations
• Background in real estate, property management, or multi-source ERP data environments (Yardi, Entrata, RealPage)
• Experience with Cosmos DB, Azure SQL, or similar operational data stores alongside lakehouse platforms
• Knowledge of data governance frameworks, data lineage tooling, and metadata management within Unity Catalog
• Background in legacy BI migration or platform modernization programs

Benefits

• A high-impact role at the center of Greystar’s enterprise data transformation
• Collaborative, engineering-driven team culture with a strong focus on craft, automation, and continuous improvement
• Access to cutting-edge tooling — Databricks, full Azure stack, GitHub Enterprise, and an active AI innovation agenda
• Flexible work arrangements
• Opportunity to define the DataOps discipline and lead Greystar’s self-healing pipeline and agentic AI roadmap
• In addition to the base salary, this role may be eligible to participate in a quarterly or annual bonus program based on individual and company performance.
• Competitive Medical, Dental, Vision, and Disability & Life insurance benefits. Low (free basic) employee Medical costs for employee-only coverage; costs discounted after 3 and 5 years of service.
• Generous Paid Time off. All new hires start with 15 days of vacation, 4 personal days, 10 sick days, and 11 paid holidays. Plus your birthday off after 1 year of service! Additional vacation accrued with tenure.
• For onsite team members, onsite housing discount at Greystar-managed communities are available subject to discount and unit availability.
• 6-Week Paid Sabbatical after 10 years of service (and every 5 years thereafter).
• 401(k) with Company Match up to 6% of pay after 6 months of service.
• Paid Parental Leave and lifetime Fertility Benefit reimbursement up to $10,000 (includes adoption or surrogacy).
• Employee Assistance Program.
• Critical Illness, Accident, Hospital Indemnity, Pet Insurance and Legal Plans.
• Charitable giving program and benefits.
• Benefits offered for full-time employees.
• This position may be performed remotely anywhere within the United States except the state of Alaska.

Company Overview

Company H1B Sponsorship