Data Engineer-Data Platforms-Azure
IBM
IBM
Join our dynamic team at IBM Consulting, where you'll collaborate with leading companies worldwide to shape their hybrid cloud and AI strategies. We foster a culture of curiosity and innovation, empowering you to challenge the status quo and deliver impactful solutions. Your long-term career development is our priority, valuing your unique contributions and experiences.
This role focuses on Data Engineering within the Azure Cloud ecosystem. You will be instrumental in advising on, developing, and maintaining sophisticated data engineering solutions, ensuring the seamless operation of data platforms.
Design, build, and operate robust batch and real-time data pipelines using key Azure services including Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hubs.
Develop and maintain the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.
Implement and manage Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (with Python), Azure Purview, and Cosmos DB.
Leverage open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala, to enhance data engineering solutions.
Ensure the continuous and smooth operation of all data engineering solutions for efficient batch and real-time data processing.
Proven experience in designing, building, and operating batch and real-time data pipelines on the Azure Cloud. This includes hands-on work with Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hubs.
Demonstrated ability in developing the data layer using Azure Synapse Analytics, SQL DW, and Cosmos DB.
Proficiency with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB.
Experience with open-source technologies such as Apache Airflow and dbt, coupled with strong skills in Spark/Python or Spark/Scala for data engineering tasks.
IBM
Data Analytics