Spark Developer
Infosys
Infosys
Join a dynamic, data-centric team focused on transforming complex datasets into actionable insights through distributed processing. This role involves close collaboration with engineers, analysts, and stakeholders to build robust and scalable data solutions using Spark. Your contributions will directly impact faster decision-making and enhance customer outcomes. We encourage taking ownership, driving performance improvements, and clarifying ambiguous challenges through structured analysis and thoughtful implementation. If you thrive on solving large-scale data problems, optimizing pipelines, and working in a learning-oriented environment, this position offers significant growth opportunities and a chance to make a visible impact.
Key responsibilities include designing, developing, and maintaining scalable Spark data processing jobs for batch and near-real-time workloads. Analyze vast datasets to identify trends, anomalies, and data quality issues, implementing validation and reconciliation checks. Optimize Spark applications for peak performance by tuning partitions, caching strategies, memory usage, and execution plans. Collaborate with cross-functional teams to translate business needs into technical solutions and well-defined deliverables. Implement robust error handling, logging, and monitoring for reliability and easier troubleshooting. Engage in code reviews, adhere to engineering best practices, and contribute to reusable components and standards. Support deployments and production issues by conducting root-cause analysis and implementing preventive fixes.
Essential qualifications include a Bachelor’s degree (or equivalent) in Engineering, Technology, Computer Science, or a related field (B.Tech/BE/MSc or equivalent), coupled with 5+ years of experience in data engineering or big data processing. Proven hands-on experience in building and maintaining Spark-based data processing solutions is critical, alongside a strong understanding of distributed computing concepts and data processing fundamentals. The ability to work independently on assigned modules while collaborating effectively within a team is also required. A Master’s degree (M.Tech/MCA or equivalent) in a relevant discipline is preferred, along with proven experience delivering end-to-end Spark pipelines, including development, testing, and production support. Experience in enhancing job performance and stability through Spark tuning and structured troubleshooting practices is beneficial, as is familiarity with building reusable frameworks/components for standardized Spark development. Strong communication skills to articulate technical trade-offs and align solutions with stakeholder expectations are highly valued.
Infosys
Information Technology & Services