Pyspark Developer
Infosys
Infosys
Join Infosys as a PySpark Developer and contribute to next-generation digital services. This role within the Data & Analytics unit focuses on building scalable solutions using cutting-edge technologies for global clients. You'll engage with modern analytics ecosystems and large-scale data modernization programs.
This position offers continuous learning opportunities, professional certifications, and a clear path for career advancement within Infosys's thriving data and analytics practice. Be a part of a global leader shaping the future of data.
The primary focus is on developing and optimizing data processing pipelines, ensuring efficiency and scalability. You will collaborate with cross-functional teams to deliver high-quality data solutions.
Design and develop large-scale, distributed data processing pipelines using PySpark and Hadoop.
Leverage expertise in Spark Core, Spark SQL, Batch processing, and Spark Streaming for robust data manipulation.
Work with Hadoop, HDFS, and Hive, alongside other Big Data technologies.
Apply Data Warehousing and ETL concepts to enhance data management strategies.
Utilize UNIX shell scripting for efficient job scheduling and application execution.
Participate actively in project development life cycle, from conception to maintenance.
Translate business requirements into technical specifications and solutions.
A minimum of 2-3 years of hands-on experience developing scalable solutions with PySpark is essential.
Demonstrated expertise in PySpark, Spark, Hadoop, SQL, Hive, Sqoop, and UNIX shell scripting is required.
Possess strong knowledge of Spark architecture and transformations using Spark and PySpark.
Familiarity with Agile methodologies, including daily stand-ups, sprint planning, and retrospectives, is expected.
Excellent problem-solving and analytical skills are crucial for troubleshooting and resolving complex data issues.
A Bachelor's or Master's degree in a relevant field is preferred.
Ability to collaborate effectively within a diverse, multi-stakeholder environment.
Infosys
Information Technology & Services