PySpark Developer
Infosys
Infosys
Seeking a skilled PySpark Developer with 5-9 years of experience to build and optimize large-scale data processing applications. This role involves interfacing with clients, understanding requirements, designing solutions, and ensuring high customer satisfaction within the Infosys delivery team.
Contribute to building efficient programs and systems, guiding your team towards high-quality code. This is an excellent opportunity to help clients navigate their digital transformation journey, leveraging your expertise in data and analytics.
Design, develop, and maintain scalable data pipelines utilizing PySpark. Develop robust ETL/ELT processes for data ingestion, transformation, and loading. Work with extensive datasets in distributed computing environments. Optimize Spark jobs for peak performance, scalability, and reliability. Create data processing solutions using Spark SQL, DataFrames, and RDDs. Integrate data from diverse sources like databases, APIs, and cloud platforms.
Collaborate effectively with data engineers, architects, and business stakeholders. Troubleshoot and conduct root cause analysis for production issues. Uphold data quality, governance, and security standards.
A minimum of 5-9 years of experience in Data Engineering or Big Data Development is essential. Demonstrate strong hands-on proficiency with PySpark and Python. Possess extensive experience with Apache Spark, Spark SQL, DataFrames, and ETL development. Exhibit good knowledge of the Hadoop Ecosystem, including Hive, HDFS, and YARN. Strong SQL and database skills are required. Experience working within Unix/Linux environments is necessary. Familiarity with data warehousing concepts is expected.
Infosys
IT Consulting