PySpark Developer

Infosys

5–9 yrs Bengaluru Full Time Work from office
Infosys logo
Posted : today
Actively hiring

Job description

Seeking a skilled PySpark Developer with 5-9 years of experience to build and optimize large-scale data processing applications. This role involves interfacing with clients, understanding requirements, designing solutions, and ensuring high customer satisfaction within the Infosys delivery team.

Contribute to building efficient programs and systems, guiding your team towards high-quality code. This is an excellent opportunity to help clients navigate their digital transformation journey, leveraging your expertise in data and analytics.

Responsibilities

Design, develop, and maintain scalable data pipelines utilizing PySpark. Develop robust ETL/ELT processes for data ingestion, transformation, and loading. Work with extensive datasets in distributed computing environments. Optimize Spark jobs for peak performance, scalability, and reliability. Create data processing solutions using Spark SQL, DataFrames, and RDDs. Integrate data from diverse sources like databases, APIs, and cloud platforms.

Collaborate effectively with data engineers, architects, and business stakeholders. Troubleshoot and conduct root cause analysis for production issues. Uphold data quality, governance, and security standards.

Qualifications

A minimum of 5-9 years of experience in Data Engineering or Big Data Development is essential. Demonstrate strong hands-on proficiency with PySpark and Python. Possess extensive experience with Apache Spark, Spark SQL, DataFrames, and ETL development. Exhibit good knowledge of the Hadoop Ecosystem, including Hive, HDFS, and YARN. Strong SQL and database skills are required. Experience working within Unix/Linux environments is necessary. Familiarity with data warehousing concepts is expected.

Essential Skills

PySparkSpark SQLPythonETL DevelopmentBig DataApache SparkDataFramesRDDsHadoop EcosystemHiveHDFSYARNSQLUnix/LinuxData Warehousing

Highlights

  • Actively hiring

More Details

RolePySpark Developer
DepartmentData Engineering, Data & Analytics
Employment TypeFull Time, Work from office

About the Company

Infosys logo

Infosys

IT Consulting

PySpark Developer at Infosys | SkillMX | SkillMX