Seeking a Lead Principal Core Infrastructure Engineer to provide senior technical leadership for OCI’s network monitoring and observability platform. This role is pivotal in managing systems that offer critical visibility into the health, performance, and behavior of OCI's network infrastructure. You will architect robust, scalable, and efficient distributed systems for collecting, processing, and analyzing vast amounts of network telemetry.

Key responsibilities include defining the technical strategy and engineering standards for telemetry pipelines, real-time stream processing, metrics, alerting, visualization, and network health analytics. Drive the architecture for cloud-scale systems demanding high availability, data integrity, low latency, and operational efficiency. This position involves identifying and resolving systemic performance and reliability constraints across multiple services and engineering teams.

As a senior technical authority, you will lead complex architectural and production issues, influence cross-organizational technical decisions, and mentor senior engineers. Collaboration with network engineering, infrastructure, and service teams is essential to continuously enhance OCI’s ability to detect, understand, and resolve network-related problems.

- Define the architectural vision and technical roadmap for OCI’s network monitoring and observability platform. - Architect distributed systems for cloud-scale collection, processing, storage, and visualization of network telemetry. - Design high-volume telemetry ingestion and streaming architectures. - Establish strategies for metrics, events, alerts, topology, network state, and other critical network health signals. - Drive architectures that facilitate rapid detection, correlation, diagnosis, and isolation of network failures. - Set high standards for telemetry quality, completeness, freshness, accuracy, retention, and availability. - Lead the design of scalable and elastic systems to support OCI's infrastructure and traffic growth. - Identify and address performance, throughput, latency, storage, and scalability bottlenecks in telemetry pipelines. - Architect robust streaming and event-processing systems with effective partitioning, backpressure, and failure-handling. - Define resilient state-management, replication, and recovery strategies for distributed monitoring systems. - Evaluate architectural trade-offs regarding consistency, availability, latency, durability, and cost. - Establish SLOs and engineering standards for the observability platform's availability, durability, latency, and data freshness. - Architect fault-tolerant systems capable of continuous operation through infrastructure failures and disruptions. - Define KPIs and telemetry to monitor the health of the monitoring platform itself. - Lead diagnosis and resolution of complex production issues across networking, distributed systems, and infrastructure. - Act as a senior technical escalation point for critical incidents, driving root-cause analysis and corrective actions. - Design systems and practices for automated deployment, upgrades, and recovery with minimal customer impact. - Define methods for monitoring large-scale Layer 2 and Layer 3 network infrastructure and identifying state changes. - Drive correlation of telemetry across devices and network layers to improve fault localization and reduce resolution times. - Develop approaches for identifying abnormal network behavior, telemetry gaps, and emerging infrastructure failures. - Partner with network engineering teams to translate operational requirements into scalable monitoring capabilities. - Enhance signal quality and actionable alerting while reducing noise and operational load. - Set technical direction and influence architecture decisions across Network Monitoring and dependent OCI organizations. - Lead complex and ambiguous initiatives involving multiple systems and engineering teams. - Establish architectural patterns, engineering standards, and best practices for large-scale observability systems. - Mentor senior engineers on distributed systems, networking, reliability, and observability. - Evaluate emerging technologies to improve scale, reliability, performance, or operational efficiency. - Contribute to technical talent development through interviewing, assessment, mentoring, and knowledge sharing.

Deep expertise is required in several key areas, with sufficient breadth to lead architecture across the entire observability stack. Essential areas include:

Distributed Systems & Cloud: Large-scale distributed systems, cloud infrastructure, Kubernetes, service-oriented architectures, distributed state management, high availability, fault tolerance, capacity planning, and performance engineering.

Observability: Metrics and telemetry architecture, Prometheus, Grafana, time-series data, high-cardinality metrics, alerting, dashboards, SLOs, telemetry pipelines, and monitoring large distributed environments.

Streaming & Data Processing: Kafka, Flink, or equivalent technologies; real-time stream processing, event-driven architectures, partitioning, aggregation, backpressure, data pipelines, and large-scale telemetry processing.

Networking: Strong understanding of L2/L3 networking, routing and switching, network topology, BGP, LLDP, SNMP, gNMI, network failure modes, and network performance troubleshooting.

Network Telemetry: Experience designing or operating systems that collect and correlate telemetry from large fleets of network devices using streaming telemetry, counters, events, protocol state, and device health information.

Programming: Strong software engineering expertise in Java, Go, or comparable systems programming languages, with experience building highly concurrent, performance-sensitive production services.

Reliability Engineering: SLOs, availability and durability engineering, distributed failure handling, load shedding, throttling, rate limiting, incident response, root-cause analysis, and production readiness.

Automation: Infrastructure as Code, automated deployment and configuration management, safe rollout and rollback strategies, and operating large infrastructure fleets with minimal manual intervention.

Security: Secure multi-tenant cloud architectures, authentication and authorization, encryption, vulnerability remediation, and security considerations for infrastructure telemetry and management systems.

Experience building observability or telemetry platforms for large-scale cloud or network infrastructure is strongly preferred.