Data Engineer-Data Platforms-Azure
IBM
IBM
Embark on a fulfilling career in IBM Consulting, where you'll cultivate enduring client relationships and collaborate globally. You'll partner with industry leaders, guiding their hybrid cloud and AI transformations. Leveraging strategic alliances, robust IBM technology, and Red Hat, you'll possess the resources to drive significant change and accelerate client success.
At IBM Consulting, a spirit of curiosity fuels achievement. We encourage you to question the status quo, explore novel concepts, and develop groundbreaking solutions that yield tangible results. Our culture emphasizes continuous growth and empathy, prioritizing your long-term career development while celebrating your unique contributions.
As a Data Engineer focused on Data Platforms within the Azure ecosystem, your role involves advising on, developing, and maintaining sophisticated data engineering solutions.
You will be instrumental in designing, building, and operating both batch and real-time data pipelines, utilizing a variety of Azure services. Key responsibilities include:
- Designing and constructing robust data pipelines using Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub. - Developing and managing the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. - Implementing and maintaining critical Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (with Python), Azure Purview, and Cosmos DB. - Applying expertise in open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala, to support data engineering initiatives. - Ensuring the smooth operation and maintenance of data engineering solutions for uninterrupted batch and real-time data processing.
A strong foundation in data engineering on the Azure Cloud is essential, encompassing the design, development, and maintenance of batch and real-time data pipelines.
Demonstrated experience in the following areas is crucial:
- Proficiency with Azure services including Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for pipeline development. - Hands-on experience with Azure Data Platform components like ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. - Familiarity with open-source technologies such as Apache Airflow and dbt, alongside Spark/Python or Spark/Scala. - Expertise in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. - The ability to maintain and operate data engineering solutions to ensure seamless data processing.
IBM
Data Analytics