Spark Developer

Infosys

2–5 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Become a key player in a data-centric environment, leveraging distributed processing to transform intricate datasets into actionable insights. This role involves close collaboration with engineers, analysts, and stakeholders to develop robust, scalable data solutions powered by Spark. Your contributions will directly influence faster decision-making and enhanced customer outcomes. We encourage initiative, performance enhancement, and clear problem-solving through analytical rigor and effective implementation. If tackling large-scale data challenges, optimizing pipelines, and thriving in a collaborative, learning-focused setting appeals to you, this position offers significant growth and visibility.

Responsibilities

Design, build, and maintain scalable data processing jobs using Spark for both batch and near-real-time scenarios. Analyze extensive datasets to pinpoint trends, anomalies, and data quality concerns, incorporating validation and reconciliation mechanisms. Optimize Spark applications for peak performance through meticulous tuning of partitions, caching, memory utilization, and execution plans. Partner with cross-functional teams to translate business needs into technical specifications and concrete deliverables. Establish comprehensive error handling, logging, and monitoring systems for maximum reliability and ease of troubleshooting. Engage in code reviews, adhere to engineering best practices, and contribute to the development of reusable components and standards. Provide support for deployments and production issues by conducting root-cause analysis and implementing preventative measures.

Qualifications

A Bachelor's degree in Engineering, Technology, Computer Science, or a related field (such as B.Tech, B.E., or B.Sc.) is required, along with 3 to 5 years of hands-on experience in data engineering or big data processing. Proven expertise in building and maintaining Spark-based data processing solutions is essential. A strong grasp of distributed computing principles and data processing fundamentals is necessary. The ability to work autonomously on assigned tasks while also collaborating effectively within a team environment is key. A Master's degree (M.Tech, M.C.A., or equivalent) is preferred, as is end-to-end Spark pipeline delivery experience, including development, testing, and production support. Experience in performance and stability enhancement through Spark tuning and structured troubleshooting is advantageous, as is familiarity with creating reusable frameworks for standardized Spark development. Strong communication skills are valued for articulating technical trade-offs and aligning solutions with stakeholder expectations.

Essential Skills

SparkSpark SQLJavaData EngineeringDistributed ComputingData Processing

Good to Have

HadoopHiveKafkaAirflowDelta Lake

Highlights

  • Actively hiring

More Details

RoleSpark Developer
IndustryInformation Technology & Services, Management Consulting
DepartmentData Engineering, Data & Analytics
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

Spark Developer at Infosys | SkillMX | SkillMX