Join Disney Entertainment and ESPN Product & Technology, a dynamic team shaping the future of media through innovative technology. We're seeking a Senior Data Engineer I to enhance the reliability, scalability, and performance of Disney's Ad Technology big data platforms. This role involves hands-on development and operation of Databricks, Spark, Flink, and Airflow (MWAA) environments, ensuring robust, cost-effective, and high-performing data workflows.

This is an opportunity to contribute to a leading media technology organization that impacts millions of users globally. You'll be instrumental in building and maintaining the technological backbone for Disney's diverse media portfolio, including Disney+, Hulu, and ESPN. Embrace innovation and play a key role in developing cutting-edge products and platforms.

Key responsibilities include designing, optimizing, and maintaining Spark and Flink data pipelines within Databricks and Airflow environments, ensuring their production reliability. You will also be responsible for debugging and resolving issues in notebooks, jobs, and orchestration workflows.

Performance tuning of Spark jobs, clusters, and data pipelines for optimal performance and cost efficiency is crucial. This includes analyzing logs and workloads to identify and address bottlenecks. Proactive pipeline monitoring through dashboards and alerts using tools like Datadog, CloudWatch, or Prometheus is also part of the role.

Participate actively in incident response and root cause analysis (RCA) for pipeline issues to improve long-term stability. Collaborate closely with data engineering, BI, and platform teams, documenting best practices, contributing to runbooks, and mentoring junior engineers to foster knowledge sharing and team growth.

We are looking for a Senior Data Engineer I with at least 4 years of experience in data engineering or big data platform operations. Essential hands-on experience with Databricks, Spark, Flink, or Airflow (MWAA) in production environments is required.

Strong SQL skills and proven experience in ETL/ELT pipeline development are necessary. Familiarity with AWS services such as S3, Glue, Athena, EMR, and EKS for pipeline management, along with working knowledge of observability tools (Datadog, CloudWatch, Prometheus), is expected.

Proficiency in Python, Scala, or Java for pipeline development and automation is a must. Experience with infrastructure-as-code tools like Terraform and strong collaboration and problem-solving skills for distributed teams are also key requirements. A Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent experience, is required.