Pyspark Developer

Infosys

5–9 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Infosys is seeking a skilled PySpark Developer with 5-9 years of experience to build scalable solutions. The role involves diagnosing customer issues, designing innovative solutions, and ensuring client satisfaction within the Data & Analytics Unit.

As a Consultant in the Data & Analytics Unit, you will engage with business stakeholders to understand requirements and translate them into effective technical solutions. This position offers an opportunity to work in a global delivery environment, staying updated with the latest industry trends and technologies.

Responsibilities

Design and develop large-scale, distributed data processing pipelines using PySpark, Hadoop, and related technologies. Leverage expertise in Spark Core, Spark SQL, Batch Processing, and Spark Streaming. Work with Hadoop, HDFS, Hive, and other Big Data technologies. Apply knowledge of Data Warehousing and ETL concepts. Contribute to the development of project plans, including scoping and estimation. Collaborate with Solution Designers and business SMEs to understand and document system requirements. Participate actively in an Agile/Scrum environment, including daily stand-ups, sprint planning, reviews, and retrospectives.

Qualifications

A minimum of 5-9 years of experience in designing and developing distributed data processing pipelines. Strong expertise in PySpark, Spark Core, Spark SQL, Batch processing, and Spark Streaming. Proficiency with Hadoop, HDFS, Hive, and related Big Data technologies. Familiarity with Data Warehousing and ETL concepts is essential. Experience with UNIX shell scripting is highly desirable for job scheduling and execution. A Bachelor of Engineering degree is required. Excellent problem-solving, analytical, communication, and collaboration skills are necessary to work effectively in diverse teams and with multiple stakeholders.

Essential Skills

PySparkSparkHadoopSQLHiveSqoopUNIX Shell ScriptingSpark CoreSpark SQLBatch ProcessingSpark StreamingHDFSETLData WarehousingAgileScrum

Good to Have

UNIX shell scripting

Highlights

  • Actively hiring

More Details

RolePyspark Developer
IndustryInformation Technology & Services
DepartmentData Engineering, Consulting
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

Pyspark Developer at Infosys | SkillMX | SkillMX