Spark
Infosys
Infosys
Join a forward-thinking, data-centric team where your expertise in distributed processing will transform complex datasets into actionable insights. In this role, you will partner closely with engineers, analysts, and stakeholders to build robust and scalable data solutions utilizing Spark. Your contributions will directly influence faster decision-making and enhance customer outcomes.
We encourage a proactive approach, empowering you to take ownership of projects, drive performance improvements, and bring clarity to intricate problem statements through meticulous analysis and thoughtful execution. If you thrive on tackling large-scale data challenges, optimizing pipelines, and growing within a collaborative environment that champions learning and continuous advancement, this opportunity offers significant scope for both technical development and impactful contributions.
Design, develop, and maintain scalable data processing jobs for batch and near-real-time workloads using Spark. Analyze extensive datasets to pinpoint trends, anomalies, and data quality concerns, implementing comprehensive validation and reconciliation checks. Enhance Spark application performance through meticulous tuning of partitions, caching, memory management, and execution plans. Collaborate with cross-functional teams to translate business needs into effective technical solutions and clearly defined deliverables. Implement resilient error handling, logging, and monitoring systems to ensure operational reliability and simplify troubleshooting. Actively participate in code reviews, adhere to best engineering practices, and contribute to the development of reusable components and standards. Provide support for deployments and production incidents, conducting root-cause analyses and implementing preventative measures.
Possess a Bachelor’s degree (or equivalent) in Engineering, Technology, Computer Science, or a related field (e.g., B.Tech, B.E., M.Sc.). Gain 3–5 years of experience in data engineering or big data processing initiatives. Demonstrate hands-on expertise in building and maintaining Spark-based data processing solutions. Exhibit a strong grasp of distributed computing principles and fundamental data processing concepts. Showcase the ability to manage assigned modules independently while collaborating effectively within a team.
Preferred qualifications include a Master’s degree (M.Tech, MCA, or equivalent) in a relevant discipline, proven end-to-end Spark pipeline delivery experience, and expertise in optimizing Spark job performance and stability. Familiarity with building reusable frameworks and strong communication skills are also advantageous.
Infosys
IT Consulting