Data Engineer-Data Platforms-AWS
IBM
IBM
Advance your career in Data Platforms with a focus on AWS. This role involves designing, building, and maintaining robust, scalable, and secure data solutions. You'll collaborate closely with architects, application teams, and business stakeholders to deliver enterprise-grade data services. We foster a culture of curiosity and continuous learning, encouraging innovation to drive meaningful client impact.
We seek a seasoned AWS Data Engineer with proven experience in building high-performance data platforms. A strong command of AWS services, distributed data processing frameworks, and SQL-driven analytics is essential for success in this position.
Develop and manage data pipelines and platforms within the AWS ecosystem. Construct and operate serverless and containerized applications utilizing AWS Lambda, EC2, and EKS. Create solutions for batch and large-scale data processing with Apache Spark. Implement data ingestion, transformation, and analytics workflows using AWS Glue and Athena. Write complex, high-performance SQL queries for analytics, reporting, and data validation purposes. Manage and optimize data storage solutions leveraging AWS S3 and RDS. Utilize Infrastructure as Code (IaC) with CloudFormation for AWS infrastructure management. Implement robust AWS security practices, including IAM, VPC, data encryption, and access controls. Write clean, scalable, and maintainable code in Python and/or Scala. Monitor, troubleshoot, and optimize the performance, cost, and reliability of data workloads. Collaborate with cross-functional teams to support analytics, reporting, and regulatory requirements.
Mandatory qualifications include extensive hands-on experience with core AWS services such as S3, Lambda, EC2, EKS, AWS Glue, Athena, RDS, VPC, IAM, and CloudFormation. Proficiency in writing complex SQL queries, including joins, aggregations, user-defined functions, and performance tuning. A solid understanding of AWS Security, encompassing IAM policies, VPC security, encryption, and least-privilege access principles. Demonstrated programming experience with Python and/or Scala. Significant experience with Apache Spark for large-scale data processing is required. Effective communication skills are necessary to articulate technical concepts to a diverse audience. A strong foundation in data engineering principles, including ETL/ELT, data modeling, and distributed systems, is essential.
IBM
Software