Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem. You'll leverage cutting-edge Google services to create efficient batch and real-time data pipelines, manage data migration, and optimize data layer design. We foster a culture of curiosity and innovation, encouraging you to challenge norms and create impactful solutions.
As a Data Engineer, you will be instrumental in developing and implementing data strategies that drive significant client impact. Your expertise will be key in transforming how organizations utilize their data, contributing to their digital transformation journeys.
Design and develop scalable batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.
Build and maintain data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.
Manage and schedule data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow).
Implement effective data migration strategies to ensure seamless data transfer.
Optimize data layer design using BigQuery, BigTable, and Cloud Spanner for efficient storage and retrieval.
Proven experience designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem.
Expertise in developing and managing batch and real-time data pipelines for Data Warehouse and Datalake using Google services and open-source technologies (e.g., Apache Airflow, dbt, Spark/Python, Spark/Scala).
Proficiency with Google Cloud services including Cloud Storage, BigTable, BigQuery, DataProc, and DataFlow.
Familiarity with data platform management, including scheduling and orchestration tools like Cloud Scheduler and Cloud Composer (Airflow).
Experience in data layer design and optimization using services like BigQuery, BigTable, and Cloud Spanner.
IBM
IT Consulting