Join our team as a Senior Core Infrastructure Engineer, a pivotal role focused on designing, implementing, and optimizing robust distributed systems. You will be instrumental in enhancing scalability, resiliency, and operability while ensuring seamless feature delivery and rigorous load/performance testing. Your expertise will be crucial in leveraging advanced data plane platforms and distributed state tools for high-volume data processing and retrieval. This position also involves meticulous peer code reviews to uphold scalability standards and building resilient fault-tolerant systems through redundancy, replication, and automatic failover.

We are seeking an engineer who excels in applying recovery-oriented principles, implementing effective retry mechanisms, circuit breakers, and timeouts. Proactive issue detection and mitigation using tests, alarms, dashboards, and telemetry are key. You will also author essential runbooks and actively participate in incident response and root cause analyses. The role requires proficiency in implementing standard replication and synchronization, developing automation and Infrastructure as Code (IaC) for troubleshooting and maintenance, and applying advanced security controls like encryption and access management, all while adhering to strict change, compliance, and documentation standards.

Key responsibilities include designing and implementing components for horizontally and vertically scalable distributed systems, optimizing code for large-scale data processing, and ensuring scalability requirements are met through implementation and review. You will utilize data plane platform components for large-scale data handling and conduct performance and load testing.

Develop fault-tolerant components with redundancy, replication, and automatic failover, applying recovery-oriented computing principles to handle service disruptions. Implement retry mechanisms, circuit breakers, and timeouts to manage network unreliability. Proactively detect and address issues through tests and alarm configurations, supporting failure recovery by creating and executing runbooks. Build and customize dashboards, telemetry, and alerting systems for monitoring component health.

Design and implement functional requirements and tests for assigned features, evaluating system correctness with fault-injection and brown-out scenarios. Implement standard data replication and synchronization techniques for data integrity and availability. Diagnose, debug, and resolve system component issues, implementing strategies to prevent interruptions and eliminate the need for maintenance windows. Design and implement automation scripts for troubleshooting operational issues and participate in operational support rotations, including incident response and root cause investigations.

Apply advanced security measures such as encryption and access controls for multi-tenant environments. Implement remediation plans to enhance security continuously. Ensure cloud infrastructure complies with industry standards and regulations, maintaining up-to-date documentation. Maintain automation scripts and tools (e.g., IaC) for cloud infrastructure management and adhere to change management plans for patching, updating, and rolling back applications.

This senior role requires a proactive approach to system design, focusing on scalability, reliability, and operational excellence within distributed systems. You will be expected to independently identify and address complex technical issues, analyze data from multiple sources for troubleshooting, and contribute to knowledge sharing and best practices.

Key duties involve tracking project timelines with minimal supervision, prioritizing work effectively, and collaborating across teams to align expectations and achieve shared objectives. Building a comprehensive understanding of business and stakeholder needs is crucial for effective partnerships. Active listening and embracing diverse perspectives are highly valued.

We encourage continuous learning, expecting you to actively build new skills and stay current with industry trends. Seeking and leveraging feedback for improvement is essential, alongside contributing to a culture of knowledge sharing. Furthermore, you will be tasked with developing ideas and recommending updates to enhance team processes, protocols, and workflows, actively seeking input from colleagues on alternative approaches for continuous improvement.