Senior PySpark Developer

Infosys

8–11 yrs Bengaluru Full Time Work from office
Infosys logo
Posted : today
Actively hiring

Job description

Join Infosys Limited as a Principal Consultant in our Data & Analytics Unit, based in Bangalore, India. This full-time, onsite position focuses on advanced Data Engineering and Big Data technologies. You will leverage your extensive experience to drive innovation and deliver impactful data solutions.

This role offers a unique opportunity to work on complex data challenges, design cutting-edge data architectures, and contribute significantly to our clients' data strategies. Become a key player in transforming data into actionable insights and driving business value.

Responsibilities

Design and implement robust, scalable data pipelines using PySpark and Apache Spark. Develop and refine ETL/ELT processes to handle large volumes of diverse data. Optimize Spark applications for peak performance, focusing on Spark SQL, DataFrames, and RDDs. Collaborate closely with data architects and business stakeholders to accurately capture data requirements.

Implement rigorous data quality, validation, and governance protocols. Tune Spark jobs for optimal performance, memory efficiency, and scalability. Seamlessly integrate data from various sources, including databases, APIs, cloud storage, and streaming platforms. Provide essential support for production deployments and efficiently resolve any performance bottlenecks. Mentor junior developers and actively participate in code review sessions to foster best practices.

Qualifications

We are seeking a Senior PySpark Developer with a minimum of 8 years of IT experience and a proven track record in Data Engineering and Big Data technologies. Essential expertise includes PySpark, Python, Spark SQL, ETL development, Data Warehousing, and cloud platforms like AWS, Azure, or GCP. A strong understanding of SQL and database concepts, along with experience in the Hadoop Ecosystem (Hive, HDFS, YARN), is required.

Familiarity with Data Warehousing principles and Data Modeling techniques is crucial. Hands-on experience with Unix/Linux environments is also a prerequisite. A Master's degree (M.Tech) or a Bachelor's degree (B.E./B.Tech) is expected for this role.

Essential Skills

PySparkPythonSpark SQLETL DevelopmentData WarehousingData ModelingHadoop EcosystemHiveHDFSYARNSQLUnix/LinuxAWSAzureGCP

Highlights

  • Actively hiring

More Details

RoleSenior PySpark Developer
DepartmentData Engineering, Data & Analytics
Employment TypeFull Time, Work from office

About the Company

Infosys logo

Infosys

IT Consulting

Senior PySpark Developer at Infosys | SkillMX | SkillMX