Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting as a Data Engineer specializing in Azure Data Platforms. You will play a crucial role in designing, developing, and maintaining robust data engineering solutions within the Azure cloud ecosystem. This position offers the opportunity to collaborate with leading companies, shaping their hybrid cloud and AI strategies.
Our culture fosters curiosity and innovation, empowering you to challenge the status quo and create impactful solutions. We are committed to your long-term career development, valuing your unique contributions and experiences in a supportive and empathetic environment.
Design, build, and operate efficient batch and real-time data pipelines leveraging a variety of Azure services. This includes developing the data layer on platforms like Azure Synapse Analytics, SQL DW, and Cosmos DB. You will apply your expertise in Azure Data Platform components, integrating open-source technologies such as Apache Airflow and dbt, alongside Spark/Python or Spark/Scala for data engineering solutions.
Operate key Azure services like Event Hub, Streaming Analytics, Managed Streaming for Apache Kafka, and Azure Databricks with Spark. Your work will directly contribute to building and maintaining scalable and performant data infrastructure.
Proficiency in Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark and SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. Experience in designing, building, and operating batch and real-time data pipelines using Azure services is essential.
Familiarity with open-source technologies like Apache Airflow and dbt, as well as Spark/Python or Spark/Scala, is required. You should also have experience operating Azure services such as Event Hub, Streaming Analytics, Managed Streaming for Apache Kafka, and Azure Databricks with Spark.
IBM
Data Analytics