Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies globally. Leverage robust IBM technology and Red Hat to drive meaningful change and accelerate client impact. We foster a culture of curiosity, innovation, and continuous growth, where your unique skills are valued for long-term career development.
This role focuses on Data Engineering within Data Platforms on Azure, advising on, developing, and maintaining cutting-edge data solutions.
As a Data Engineer on Azure, you will be instrumental in designing, building, and operating robust batch and real-time data pipelines. This includes developing the data layer using services like Azure Synapse Analytics and Cosmos DB. You will implement and maintain key Azure Data Platform components such as ADLS2, Blob Storage, and Azure Purview, ensuring seamless data processing.
Your responsibilities will also involve leveraging open-source technologies like Apache Airflow and dbt, alongside Spark/Python or Spark/Scala, to enhance data engineering solutions. Ensuring the operational efficiency and maintenance of these data pipelines is crucial for delivering reliable real-time and batch data processing.
We are seeking individuals with extensive experience in data engineering on the Azure Cloud ecosystem. Proven ability to design, build, and operate batch and real-time data pipelines using Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub is essential.
Demonstrated experience in developing the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is required. Proficiency with Azure Data Platform components including ADLS2, Blob Storage, and Azure Functions with Python is expected. Familiarity with open-source technologies such as Apache Airflow, dbt, and Spark/Python or Spark/Scala is highly desirable for supporting advanced data engineering solutions.
IBM
Technology