Senior PySpark Developer
Infosys
Infosys
Join Infosys Limited as a Principal Consultant in our Data & Analytics Unit, based in Bangalore, India. This full-time, onsite position focuses on advanced Data Engineering and Big Data technologies. You will leverage your extensive experience to drive innovation and deliver impactful data solutions.
This role offers a unique opportunity to work on complex data challenges, design cutting-edge data architectures, and contribute significantly to our clients' data strategies. Become a key player in transforming data into actionable insights and driving business value.
Design and implement robust, scalable data pipelines using PySpark and Apache Spark. Develop and refine ETL/ELT processes to handle large volumes of diverse data. Optimize Spark applications for peak performance, focusing on Spark SQL, DataFrames, and RDDs. Collaborate closely with data architects and business stakeholders to accurately capture data requirements.
Implement rigorous data quality, validation, and governance protocols. Tune Spark jobs for optimal performance, memory efficiency, and scalability. Seamlessly integrate data from various sources, including databases, APIs, cloud storage, and streaming platforms. Provide essential support for production deployments and efficiently resolve any performance bottlenecks. Mentor junior developers and actively participate in code review sessions to foster best practices.
We are seeking a Senior PySpark Developer with a minimum of 8 years of IT experience and a proven track record in Data Engineering and Big Data technologies. Essential expertise includes PySpark, Python, Spark SQL, ETL development, Data Warehousing, and cloud platforms like AWS, Azure, or GCP. A strong understanding of SQL and database concepts, along with experience in the Hadoop Ecosystem (Hive, HDFS, YARN), is required.
Familiarity with Data Warehousing principles and Data Modeling techniques is crucial. Hands-on experience with Unix/Linux environments is also a prerequisite. A Master's degree (M.Tech) or a Bachelor's degree (B.E./B.Tech) is expected for this role.
Infosys
IT Consulting