Join our dynamic team as a Staff Engineer specializing in Data Engineering. This role is pivotal in shaping our data infrastructure, focusing on building robust and scalable data pipelines. You will leverage cutting-edge technologies to manage and process vast datasets, ensuring data integrity and accessibility for various business needs.

We are seeking a seasoned professional who can drive innovation in data architecture and contribute to high-impact data solutions. Your expertise will be crucial in optimizing data flows and supporting data-driven decision-making across the organization.

Design, develop, and maintain scalable data pipelines using PySpark for processing large datasets from diverse sources. Integrate data from multiple sources, ensuring high data quality and reliability. Optimize data processing jobs for peak performance and cost-efficiency. Collaborate closely with data scientists, analysts, and stakeholders to define data requirements and deliver effective solutions. Develop and maintain ETL/ELT processes for seamless data loading into data warehouses and data lakes. Implement robust data validation and monitoring to guarantee data accuracy and consistency. Create comprehensive documentation for data engineering processes and best practices. Proactively identify, troubleshoot, and resolve data-related issues.

A Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline is required. Possess 3-8 years of professional experience in data engineering or a similar technical domain. Demonstrate proficiency in PySpark and Python, alongside strong knowledge of big data technologies like Hadoop, Hive, and Spark. Experience with major cloud platforms (AWS, Azure, GCP) and their associated data services is essential. Familiarity with data warehousing solutions such as Amazon Redshift, Google BigQuery, and Snowflake is expected. Knowledge of both relational (e.g., MySQL) and NoSQL databases (e.g., MongoDB, Cassandra) is beneficial. Experience with ETL/ELT processes and data pipeline orchestration tools like Apache Airflow or Apache NiFi is highly valued. Exceptional analytical, problem-solving, and communication skills are a must.