Software Engineer - CLI Tools Building

Role : Software Engineer - CLI Tools & Data Pipelines Location: USA Remote (EST or PST) In this role, you will first build automated tooling and a regression test harness to continuously evaluate the quality of our PII Rewrite Service. Alongside this, you will design a fault-resilient command-line tool to automate the creation of Owned Test Accounts (OTAs) and orchestrate the upload of massive amounts of sanitized data from our internal storage (CNS) using AI. KEY RESPONSIBILITIES: Golden Dataset & Auto-Evaluation Pipeline Dataset Conversion Tooling: Build the conversion tools required to transform human-rater annotated evaluation results into a structured, machine-interpretable "Golden Dataset" using Protocol Buffers. Regression Test Harness: Build an Auto-Eval Harness that treats this dataset as a regression test suite to automatically evaluate the Rewrite Service and calculate key metrics like PII Recall, PII Precision, and Rewrite Consistency. Multi-Format & Multi-Language Scaling: Ensure the extraction pipeline and evaluation harness support core Workspace assets (Docs, Sheets, Slides, Gmail, PDFs, MS Office, Calendar, Chats) across multiple languages (English, Spanish, Portuguese, Japanese, Korean). OTA Provisioning & Data Ingestion Automation: CLI Tool Development: Develop the smb ota_provisioner, a robust command-line tool (in C++, Python, or Go) to orchestrate OTA check, creation, and data upload workflows. Automated Provisioning: Interface with backend systems via Stubby (e.g., TestaccountService) to dynamically create and assign Prod Gaia OTAs and Workspace (Dasher) accounts. High-Volume API Ingestion: Ingest sanitized SMB data from Colossus (CNS) and upload it to the OTAs in parallel by leveraging and extending the internal ota_lifecycle tool. Rate Limiting & Resilience: Implement robust error handling and retry policies with exponential backoff to respect strict write limits on downstream APIs (Gmail, Drive). Minimum Qualifications: Strong programming experience in C++, Python, or Go. Experience building robust, fault-tolerant Command Line Interface (CLI) tools and data processing pipelines. Familiarity with defining and parsing structured data schemas (e.g., Protocol Buffers, JSON, TextFormat). Experience working with RPC frameworks (e.g., Stubby/gRPC) and REST APIs. Solid understanding of system design principles, particularly around handling API rate limits and implementing exponential backoff mechanisms.