Join our team as a Staff Data Engineer and play a pivotal role in shaping our data infrastructure. You will be instrumental in designing, building, and maintaining robust data pipelines that process vast amounts of information. This role offers a fantastic opportunity to work with cutting-edge big data technologies and cloud platforms, contributing directly to our data-driven decision-making processes.

This position requires a seasoned professional with a solid background in data engineering principles and practices. You will collaborate with cross-functional teams, including data scientists and analysts, to deliver high-quality data solutions that meet evolving business needs.

Design, develop, and maintain scalable data pipelines using PySpark to handle large-scale data from diverse sources. Integrate data from multiple origins, ensuring exceptional data quality and reliability for all downstream applications. Optimize data processing jobs for peak performance and cost-effectiveness, maximizing resource utilization. Collaborate closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver impactful data solutions. Develop and maintain robust ETL processes, meticulously extracting, transforming, and loading data into our data warehouses and data lakes. Implement comprehensive data validation and monitoring procedures to guarantee data accuracy and consistency across all systems. Create detailed documentation for data engineering processes, workflows, and established best practices. Proactively identify, troubleshoot, and resolve data-related issues in a timely manner.

A minimum of 3 years of professional experience in data engineering or a closely related technical field is essential. A Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related discipline is required. Demonstrated proficiency in PySpark and Python programming languages. Strong command of big data technologies, including Hadoop, Hive, and Spark ecosystems. Hands-on experience with major cloud platforms such as AWS, Azure, or GCP, and their associated data services. Familiarity with modern data warehousing solutions like Amazon Redshift, Google BigQuery, or Snowflake. Knowledge of both relational (e.g., MySQL) and NoSQL databases (e.g., MongoDB, Cassandra). Proven experience with ETL/ELT processes and data pipeline orchestration tools like Apache Airflow or Apache NiFi. Excellent analytical and problem-solving capabilities to tackle complex data challenges. Superior verbal and written communication skills, with the ability to articulate technical concepts to diverse audiences.