Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and embark on a career focused on building lasting client relationships and collaborating globally. You will partner with leading organizations across various sectors, guiding them through their hybrid cloud and AI transformations. Leveraging IBM technology, Red Hat, and support from strategic partners, you'll have the resources to drive significant change and accelerate client success.
At IBM Consulting, curiosity is key to achievement. We encourage innovative thinking, exploration of new concepts, and the development of groundbreaking solutions that deliver tangible results. Our supportive and growth-oriented culture prioritizes your long-term career development while celebrating your unique contributions.
As a Data Engineer specializing in Azure Data Platforms, your role involves advising on, developing, and maintaining robust data engineering solutions within the Azure Cloud ecosystem. You will be instrumental in designing, building, and operating efficient batch and real-time data pipelines utilizing a variety of Azure services.
Key responsibilities include: - Designing and constructing data pipelines using Azure services like Synapse Analytics, Data Factory, Databricks, and Event Hub. - Developing and managing the data layer across Azure Synapse Analytics, SQL DW, and Cosmos DB. - Implementing and maintaining critical Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark & SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. - Applying expertise in open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala, to enhance data engineering solutions. - Ensuring the seamless operation and maintenance of data engineering solutions for efficient batch and real-time data processing.
This role requires proficiency and experience in several key areas:
Azure Data Platform Expertise: Demonstrated experience with Azure services including Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for building batch and real-time data pipelines. Familiarity with Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark & SQL), Azure Functions (Python), Azure Purview, and Cosmos DB is essential.
Open Source and Programming Skills: Practical application of open-source technologies like Apache Airflow and dbt. Strong skills in Spark with Python or Scala are also required to support advanced data engineering solutions.
Data Pipeline and Data Layer Development: Proven experience in developing and maintaining data engineering solutions on Azure, encompassing the design, build, and operation of both batch and real-time data pipelines. Experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is crucial.
IBM
Data Analytics