Shape your future with confidence at EY by joining our Global Delivery Services team. As a Lead AWS Data Engineer Manager/Architect, you will provide technical leadership and hands-on execution for enterprise data platforms on Amazon Web Services (AWS). This role is crucial for designing, migrating, modernizing, and operating cloud-native data architectures that support critical financial, advisor payout, mobility, and corporate analytics platforms. You will own end-to-end technical decision-making, ensuring robust architecture, security, orchestration, and production readiness.

We are seeking individuals with deep expertise in AWS data services, Python and PySpark development, workflow orchestration, data lake design, and infrastructure as code. The ideal candidate will also possess the ability to mentor engineers and collaborate effectively with infrastructure, IAM, and database teams. This is an opportunity to contribute to a globally connected powerhouse, driving innovation and delivering exceptional client service.

As the Technical Lead and Design Authority, you will steer AWS data engineering initiatives, making architectural decisions for scalability, reliability, security, and cost optimization. You will define and enforce engineering standards, coding patterns, and operational best practices for cloud data pipelines. Your role involves providing hands-on technical guidance, conducting design reviews, and performing code reviews for data engineers.

Key responsibilities include leading the design, development, and support of cloud-native data pipelines using AWS Glue (PySpark), MWAA (Apache Airflow), and AWS Step Functions. You will drive on-premises to AWS data platform migrations, re-architecting legacy ETL processes into scalable PySpark-based solutions. Optimization of Spark workloads for performance and cost efficiency is essential. Furthermore, you will architect and implement enterprise AWS data lakes using the Medallion architecture, design and manage Apache Iceberg tables, and establish standardized ingestion and transformation patterns. Ensuring data quality, reconciliation, lineage, and auditability will be paramount. You will also lead the orchestration strategy using MWAA, design and implement Airflow DAGs, and integrate workflows with various AWS services. Security implementation, including IAM role design and encryption, along with infrastructure provisioning using Terraform, are core duties. Production readiness, root cause analysis for complex issues, and implementing logging and alerting are also key aspects of this role.

We require a Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience. Candidates should possess 8-10 years of experience in Data Engineering with strong AWS exposure, demonstrating extensive hands-on experience in designing and leading AWS data engineering solutions. Proven experience leading on-premises to AWS data platform migrations is essential.

Advanced proficiency in Python and PySpark for data processing, automation, and orchestration is mandatory, alongside strong experience with AWS Glue, Amazon S3, MWAA (Apache Airflow), Step Functions, and CloudWatch. Experience with Infrastructure as Code, preferably Terraform, and a strong understanding of data engineering principles including ETL design, data modeling, and pipeline optimization are critical. Demonstrated experience supporting enterprise, production-grade data platforms is expected. A good understanding of AI technologies, agentic systems, and tools such as Microsoft Copilot and Claude, with the ability to leverage AI-driven capabilities for solution development and delivery, is also a requirement.

Preferred domain experience includes Wealth Management, Asset Management, Capital Markets, Investment Banking, and Financial Services. Exposure to portfolio management, securities data, trade processing, risk reporting, market data, and client reporting platforms is highly desirable. Familiarity with analytics or BI platforms, Oracle databases, legacy ETL tools, and CI/CD practices are beneficial. Relevant AWS certifications are a plus.