Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and become a key player in transforming businesses through hybrid cloud and AI.
You will collaborate with leading companies worldwide, guiding them on their cloud and AI journeys. Leveraging IBM technology, Red Hat, and strategic partnerships, you'll drive significant change and deliver impactful results.
IBM Consulting fosters a culture of curiosity and innovation, encouraging you to challenge the status quo and develop creative solutions. We prioritize your career development and value your unique contributions in a supportive environment.
As a Data Engineer focused on Azure Data Platforms, your role involves designing, developing, and maintaining robust data engineering solutions.
Key responsibilities include: - Architecting and building resilient batch and real-time data pipelines using Azure services like Synapse Analytics, Data Factory, Databricks, and Event Hub. - Developing and managing the data layer within Azure Synapse Analytics, SQL DW, and Cosmos DB. - Implementing and maintaining essential Azure Data Platform components such as ADLS2, Blob Storage, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. - Utilizing open-source technologies like Apache Airflow and dbt, alongside Spark with Python or Scala, to enhance data engineering practices. - Ensuring the smooth operation and maintenance of data engineering solutions for uninterrupted batch and real-time data processing.
We are seeking candidates with demonstrated expertise in Azure services and data engineering principles.
Essential qualifications include: - Proven experience in designing, building, and operating batch and real-time data pipelines on the Azure Cloud ecosystem. - Hands-on experience with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. - Proficiency in developing the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. - Familiarity with open-source technologies such as Apache Airflow and dbt, along with Spark/Python or Spark/Scala for data engineering solutions. - Exposure to Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub is crucial for pipeline development.
IBM
Information Technology & Services