Join our team as a Data Engineer and play a pivotal role in designing, developing, and maintaining robust data solutions. You will be instrumental in the entire data lifecycle, from generation and collection to processing and transformation.

This role involves creating efficient data pipelines, ensuring the highest data quality standards, and implementing comprehensive ETL processes for seamless data migration and deployment across various systems.

Design and build scalable data pipelines for large-scale data processing. Develop and deploy ETL/ELT processes using PySpark and Spark to move and transform data. Assist in deploying solutions on major cloud platforms like AWS, Azure, or GCP. Support Big Data processing frameworks and data lakes for diverse data types. Ensure data accuracy and availability through collaboration with data scientists and analysts. Implement secure and efficient data ingestion strategies. Work with real-time streaming and batch data processing. Develop reusable code for ETL operations. Contribute to performance tuning of Spark jobs and data pipelines. Maintain governance and data security practices on cloud platforms.

A minimum of 3 years of experience is essential for this role. Candidates should have a strong command of the Databricks Unified Data Analytics Platform. Proficiency in PySpark, Spark, and cloud platforms (AWS, Azure, or GCP) is required. Familiarity with Big Data architectures, data storage, and processing concepts is expected. Solid Python coding skills for pipeline development and a good understanding of SQL are necessary. Excellent troubleshooting, debugging, and performance optimization abilities are key.