Join our Technology & Transformation practice to unlock the immense value within vast datasets. We provide strategic guidance and implementation services to help companies manage data from disparate sources, converting it into actionable information for fact-driven decision-making and gaining an insight-driven advantage. Our practice spans business intelligence, data management, performance management, and next-generation analytics including big data and cloud technologies.

We are looking for a skilled Kafka Migration Engineer to support a critical on-prem Apache Kafka to AWS MSK migration. This role involves cluster discovery, topic mapping, capacity planning, configuration, migration execution, validation, performance tuning, automation, and decommissioning. The ideal candidate will possess strong engineering experience with distributed messaging systems, a structured delivery approach, and excellent collaboration skills with application, infrastructure, and platform teams to ensure safe and scalable migrations.

Assess on-prem Kafka clusters, analyzing topics, partitions, consumer groups, and throughput to ensure migration readiness. Analyze workloads based on message volume, criticality, and latency to determine optimal MSK cluster placement. Plan and provision AWS MSK infrastructure, including brokers, storage, networking, and security. Execute UAT and production parallel testing using replication tools, validating outcomes and troubleshooting discrepancies. Tune and optimize MSK clusters for stable, scalable, and efficient performance. Collaborate with engineering teams to finalize migration plans and cutover strategies. Manage release readiness, execute production cutovers, and perform post-release validation. Support legacy Kafka cluster decommissioning after successful migration. Design and implement automation frameworks, including AI-assisted solutions, for efficient large-scale migrations. Document migration procedures, learnings, risks, and best practices to enhance our execution model.

A Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field is preferred. Requires 3-5 years of hands-on engineering experience with distributed messaging systems in a team environment. Professional proficiency in Python, Java, or similar programming/scripting languages is essential. Strong Unix/Linux fundamentals for troubleshooting are necessary. Familiarity with SDLC, CI/CD, change management, and Kubernetes deployments is beneficial.

Key technical competencies include hands-on experience with AWS services (ECS, EKS, Lambda, CloudWatch), Apache Kafka architecture, AWS MSK, and migration tooling like MirrorMaker 2. Experience with on-prem to cloud migration strategies, Kafka ecosystem components, networking/security for hybrid architectures, and programming/automation with Python, Java, or Shell is expected. Working knowledge of RESTful APIs and API Gateway patterns, CI/CD tools (GitLab, Jenkins), and observability platforms (CloudWatch, Prometheus) is also valuable.

Core competencies include exceptional analytical and troubleshooting skills, a strong ownership mindset, clear written and verbal communication, and effective collaboration across diverse engineering teams. The ability to work across time zones, build alignment, resolve issues professionally, and demonstrate integrity, sound judgment, and intellectual curiosity are critical for success.