[Remote] Site Reliability Engineer
Note: The job is a remote job and is open to candidates in USA. Johnson Technology Systems Inc is a federal government consulting firm that provides technical services to various agencies. They are seeking a Site Reliability Engineer OpenSearch to ensure the highest levels of availability and performance for mission-critical cloud services, focusing on the reliability and continuous improvement of distributed search platforms.
Responsibilities
- Provision, build, deploy, monitor, operate, and support cloud services in a globally distributed team environment
- Architect, build, deploy, and maintain high-performance OpenSearch clusters and platforms from the ground up
- Administer and optimize OpenSearch environments for high availability, resiliency, scalability, security, and performance
- Monitor and troubleshoot cluster health, node performance, indexing throughput, search latency, shard allocation, replication, and storage utilization
- Analyze and resolve operational issues, platform instability, and production incidents across infrastructure, platform, and application layers
- Conduct incident response, root cause analysis, and post-incident remediation to drive continuous improvement
- Maintain the integrity and security of servers, systems, and OpenSearch platform infrastructure
- Support platform lifecycle activities including installation, configuration, upgrades, patching, hotfixes, backup, restore, and disaster recovery
- Develop and maintain monitoring policies, alerting standards, operational runbooks, and support procedures
- Automate testing, deployment, scaling, recovery, and operational workflows for OpenSearch and related cloud services
- Ensure proper resource allocation and capacity planning across compute, memory, storage, and network resources
- Partner with product development and engineering teams to design and enhance service reliability and operational readiness
- Develop and implement testing strategies and document results for platform changes and operational improvements
- Support log ingestion, index management, retention policies, lifecycle management, and search performance tuning
- Work in a diverse environment and cross-train with other global team members
- Participate in an on-call rotation and support weekend or after-hours operational needs as required
Skills
- MUST be a US Citizen & ONLY hold US Citizenship (No Dual Citizens)
- Only submit candidates that have OpenSearch deployment ON KUBERNETES experience
- Expert with Kubernetes, including troubleshooting, operations, management, and configuration of complex Kubernetes services
- Proven hands-on expertise designing, building, deploying, supporting, and maintaining OpenSearch clusters and platforms from scratch in production environments
- Strong experience with OpenSearch administration, cluster architecture, performance tuning, scaling, upgrades, and troubleshooting
- Experience with index design, shard and replica strategy, cluster sizing, node management, snapshot/restore, backup, and disaster recovery
- Strong understanding of distributed systems, search platforms, indexing pipelines, query optimization, and high-availability architectures
- Expertise with Git
- Expertise with Concourse, including setup, management, and troubleshooting of new pipelines
- Expertise with Linux, specifically SUSE and Ubuntu
- Expertise with Kafka, Zookeeper, and Big Data technologies
- Expert in development of automation for testing, deployment, scalability, and management of cloud services
- Expertise with building, implementing, and/or supporting cloud monitoring tools
- Expert knowledge of cloud computing, infrastructure operations, and databases
- Expert understanding of web services, networking, virtualization, and internet protocols
- Ability to multitask and handle various projects, deadlines, and changing priorities
- Excellent communication and prioritization skills
- Expertise with security fundamentals as they pertain to SaaS multi-tenant application systems
- Strong interpersonal, presentation, and customer service skills
- Participation in an on-call rotation for handling P1 incidents is required
- Flexible schedule which may include weekend or after-hours work
- Ability to work effectively in a diverse, collaborative, and globally distributed team environment
- 8+ years of experience
- Experience with AWS services including Route 53, EC2, S3, CloudWatch, DynamoDB, RDS, IAM, ACM, KMS, and VPC
- Experience deploying and operating OpenSearch in AWS-based environments
- Experience with Cloud Foundry-based environments
- Experience with Jenkins, Chef, and/or Terraform
- Exposure to and understanding of troubleshooting IP networks and application stacks
- Experience with observability tools such as Prometheus and Grafana
- Experience with log ingestion pipelines, index lifecycle management, retention strategies, and search platform security controls
- Familiarity with capacity forecasting, performance benchmarking, and resilience testing for distributed search platforms
- BS/BA degree in Computer Science, Management Information Systems, or related IT discipline preferred
- Allowable substitution: An additional four (4) years of experience may be substituted for a BS/BA degree
Company Overview