Join our dynamic Product and Platform Engineering team as a Staff Engineer, focusing on building robust data solutions. This role is ideal for a passionate individual eager to contribute to scalable batch and near-real-time data pipelines. You will leverage cutting-edge technologies to drive innovation and efficiency in our data infrastructure.

We are seeking a skilled engineer to design, develop, and deploy sophisticated data processing systems. This position offers a unique opportunity to work with cloud-native services and advanced data engineering tools, playing a key role in our platform's evolution. If you are driven by technical challenges and a collaborative environment, we encourage you to apply.

Key responsibilities include designing and developing scalable batch and near-real-time ETL/ELT pipelines utilizing Snowflake and Apache Spark (PySpark, Spark SQL, Structured Streaming). You will also build structured streaming pipelines with Kafka and Spark Structured Streaming, and design dimensional data models, including Fact/Dimension and SCD Type 2.

Furthermore, you will orchestrate complex pipelines using Databricks Workflows or Apache Airflow. The role also involves integrating CI/CD pipelines with Jenkins and Git (Bitbucket/GitHub) to ensure automated deployments across development, UAT, and production environments.

Essential skills for this role include proficiency in Apache Spark (Core, Structured Streaming), PySpark, Databricks (AWS/Azure), and Advanced SQL. A strong foundation in programming languages like Python and SQL is crucial.

Experience with DevOps tools such as Jenkins and Git/GitHub/Bitbucket is required. Familiarity with cloud infrastructure, specifically AWS services like S3, EMR, EC2, IAM, and CloudWatch, is necessary. Experience with Apache Kafka is also a core requirement. Preferred qualifications include experience in financial services, payouts, or enterprise data platforms, along with hands-on expertise in Delta Lake optimization and incremental processing strategies.