Join our team as a Data Engineer and play a pivotal role in shaping our data landscape. You will be instrumental in designing, developing, and maintaining robust data solutions, focusing on efficient data generation, collection, and processing. This role involves creating and managing sophisticated data pipelines, ensuring the highest standards of data quality, and implementing effective ETL processes for seamless data migration and deployment across diverse systems. Be a key contributor to our data infrastructure and support organizational data objectives.

Develop and maintain comprehensive data solutions, including data generation, collection, and processing pipelines. Ensure the integrity and quality of data throughout its lifecycle. Implement ETL processes for effective data migration and deployment across systems. Become a subject matter expert and perform independently. Actively contribute to team discussions and problem-solving initiatives. Collaborate with cross-functional teams to define data requirements and deliver scalable solutions. Monitor and optimize data workflows for enhanced performance and reliability. Document data processes and maintain clear communication on data architecture and pipeline status. Mentor junior team members by sharing knowledge and promoting best practices.

A minimum of 3 years of experience with PySpark is essential. Proven experience in building and managing scalable data pipelines and workflows. Solid understanding of data processing frameworks and distributed computing principles. Demonstrated ability to troubleshoot and optimize complex data processing tasks. Familiarity with various data storage solutions and data integration techniques. Knowledge of performance tuning and resource management within big data environments. Requires 15 years of full-time education.