Spark Developer

Infosys

5–7 yrs Bengaluru Full Time Work from office
Infosys logo
Posted : today
Actively hiring

Job description

Join a data-driven team and leverage your expertise in distributed processing to transform complex datasets into actionable insights. This role offers the opportunity to collaborate with engineers, analysts, and stakeholders, building scalable and reliable data solutions using Spark. You'll contribute to faster decision-making and improved customer outcomes by taking ownership of deliverables, enhancing performance, and applying structured analysis to solve ambiguous problems.

This position is ideal for individuals passionate about tackling large-scale data challenges, optimizing pipelines, and thriving in a collaborative environment that fosters continuous learning and improvement. Make a visible impact across projects and teams while deepening your technical expertise.

Responsibilities

- Design, develop, and maintain scalable data processing jobs using Spark for batch and near-real-time workloads. - Analyze large datasets to identify trends, anomalies, and data quality issues, implementing validation and reconciliation checks. - Optimize Spark applications for superior performance through tuning partitions, caching strategies, memory usage, and execution plans. - Collaborate with cross-functional teams to translate business needs into well-defined technical solutions and deliverables. - Implement robust error handling, logging, and monitoring for enhanced reliability and simplified troubleshooting. - Engage in code reviews, adhere to engineering best practices, and contribute to reusable components and standards. - Support deployments and production issues by conducting root-cause analysis and implementing preventive fixes.

Qualifications

- Bachelor’s degree (or equivalent) in Engineering, Technology, Computer Science, or a related field (e.g., B.Tech/BE/MSc). - Minimum 5 years of experience in data engineering or big data processing. - Proven hands-on experience building and maintaining Spark-based data processing solutions. - Solid understanding of distributed computing concepts and data processing fundamentals. - Ability to work autonomously on assigned modules while effectively collaborating within a team. - Preferred: Master’s degree (M.Tech/MCA or equivalent) and experience with end-to-end Spark pipelines, including development, testing, and production support. - Familiarity with building reusable frameworks and strong communication skills are beneficial.

Essential Skills

SparkJavaApache SparkSpark Streaming

Good to Have

HadoopHiveKafkaAirflowDelta Lake

Highlights

  • Actively hiring

More Details

RoleSpark Developer
DepartmentData & Analytics
Employment TypeFull Time, Work from office

About the Company

Infosys logo

Infosys

IT Consulting

Spark Developer at Infosys | SkillMX | SkillMX