Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and be part of a dynamic team shaping the future of hybrid cloud and AI for leading companies worldwide. We foster a culture of curiosity, innovation, and continuous growth, empowering you to challenge the status quo and deliver impactful solutions. As a Data Engineer specializing in Data Platforms on Azure, you will play a key role in designing, developing, and maintaining robust data engineering solutions within the Azure Cloud ecosystem.
Your expertise will be instrumental in building and operating sophisticated batch and real-time data pipelines. This role offers a unique opportunity to work with cutting-edge Azure services and open-source technologies, driving significant client impact and accelerating their digital transformation journeys.
Design, build, and operate resilient batch and real-time data pipelines using essential Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub.
Develop and maintain the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB, ensuring optimal performance and scalability.
Implement and manage critical Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (Python), Azure Purview, and Cosmos DB.
Leverage proficiency in open-source technologies such as Apache Airflow and dbt, alongside Spark with Python or Scala, to enhance data engineering solutions.
Ensure the seamless operation and maintenance of data engineering solutions, guaranteeing efficient batch and real-time data processing.
Proven experience in designing, building, and operating batch and real-time data pipelines within the Azure Cloud ecosystem.
Demonstrated experience with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark and SQL), Azure Functions (Python), Azure Purview, and Cosmos DB.
Proficiency in Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for data pipeline development.
Hands-on experience with open-source technologies like Apache Airflow and dbt, coupled with expertise in Spark using Python or Scala.
Experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is essential.
IBM
Information Technology & Services