Pyspark Developer

Infosys

2–3 yrs Chennai Full Time Remote
Infosys logo
Posted : today
Actively hiring

Job description

Join Infosys as a PySpark Developer and contribute to next-generation digital services. This role within the Data & Analytics unit focuses on building scalable solutions using cutting-edge technologies for global clients. You'll engage with modern analytics ecosystems and large-scale data modernization programs.

This position offers continuous learning opportunities, professional certifications, and a clear path for career advancement within Infosys's thriving data and analytics practice. Be a part of a global leader shaping the future of data.

The primary focus is on developing and optimizing data processing pipelines, ensuring efficiency and scalability. You will collaborate with cross-functional teams to deliver high-quality data solutions.

Responsibilities

Design and develop large-scale, distributed data processing pipelines using PySpark and Hadoop.

Leverage expertise in Spark Core, Spark SQL, Batch processing, and Spark Streaming for robust data manipulation.

Work with Hadoop, HDFS, and Hive, alongside other Big Data technologies.

Apply Data Warehousing and ETL concepts to enhance data management strategies.

Utilize UNIX shell scripting for efficient job scheduling and application execution.

Participate actively in project development life cycle, from conception to maintenance.

Translate business requirements into technical specifications and solutions.

Qualifications

A minimum of 2-3 years of hands-on experience developing scalable solutions with PySpark is essential.

Demonstrated expertise in PySpark, Spark, Hadoop, SQL, Hive, Sqoop, and UNIX shell scripting is required.

Possess strong knowledge of Spark architecture and transformations using Spark and PySpark.

Familiarity with Agile methodologies, including daily stand-ups, sprint planning, and retrospectives, is expected.

Excellent problem-solving and analytical skills are crucial for troubleshooting and resolving complex data issues.

A Bachelor's or Master's degree in a relevant field is preferred.

Ability to collaborate effectively within a diverse, multi-stakeholder environment.

Essential Skills

PySparkSparkHadoopSQLHiveSqoopUNIX Shell ScriptingSpark CoreSpark SQLBatch ProcessingSpark StreamingHDFSData WarehousingETLAgile MethodologiesProblem SolvingAnalytical SkillsCommunication Skills

Highlights

  • Actively hiring

More Details

RolePyspark Developer
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Remote

About the Company

Infosys logo

Infosys

Information Technology & Services

Pyspark Developer at Infosys | SkillMX | SkillMX