Pyspark Developer

Infosys

7–9 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join Infosys as a Senior Consultant in our Data & Analytics Unit, a global leader in next-generation digital services. This role focuses on cutting-edge data engineering and analytics initiatives for international clients. You'll engage with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and large-scale enterprise data modernization programs. We foster continuous learning, offer certifications, and provide ample career growth opportunities within our dynamic data and analytics practice.

Responsibilities

Design and develop large-scale, distributed data processing pipelines using PySpark and Hadoop technologies. Leverage expertise in Spark Core, Spark SQL, Batch processing, and Spark Streaming. Work with Hadoop, HDFS, and Hive, alongside other Big Data technologies. Gain familiarity with Data Warehousing and ETL concepts. Contribute to application job scheduling and execution through UNIX shell scripting. Actively participate in the project development lifecycle, including scoping, estimating, and translating business requirements into technical solutions.

Collaborate with business stakeholders and Solution Designers to understand high-level requirements and contribute to project plans. Apply technical, business, and system knowledge to elicit detailed system requirements. Possess a strong understanding of Spark architecture and transformations. Thrive in an Agile environment, participating in daily stand-ups, sprint planning, reviews, and retrospectives. Ensure solutions meet project quality standards and collaborate with cross-functional teams to troubleshoot data issues.

Qualifications

A minimum of 7-9 years of experience is required in building scalable solutions with Pyspark. Strong proficiency in PySpark, Spark, Hadoop, and exposure to SQL, Hive, and Sqoop is essential. Experience with HDFS, Spark Core, Spark SQL, Batch processing, and Spark Streaming is critical. Familiarity with Data Warehousing and ETL concepts is expected. UNIX shell scripting skills will be an advantage for job scheduling. A minimum of 5 years of experience in project development lifecycle activities is necessary.

Candidates should possess strong problem-solving and analytical skills. Excellent verbal and written communication abilities are a must. Experience and a desire to work in a global delivery environment are important. Staying updated with new technologies and industry trends in development is encouraged. A relevant Bachelor's or Master's degree in fields such as Computer Applications, Science, Engineering, or Technology is required.

Essential Skills

PySparkHadoopSpark CoreSpark SQLBatch ProcessingSpark StreamingHDFSHiveBig Data TechnologiesSQLUNIX Shell ScriptingData WarehousingETL

Good to Have

Sqoop

Highlights

  • Actively hiring

More Details

RolePyspark Developer
DepartmentData Engineering, Data & Analytics
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

IT Consulting

Pyspark Developer at Infosys | SkillMX | SkillMX