Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting to build lasting client relationships and drive transformation in hybrid cloud and AI. As a Data Engineer focused on Google's data platforms, you'll design, develop, and maintain robust data engineering solutions within the Google Cloud ecosystem. This role involves leveraging Google services for both batch and real-time data processing, migration, and efficient data layer design, fostering innovation and delivering impactful results.
Design and build scalable batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. Develop data engineering solutions leveraging Google Cloud Storage, BigQuery, and DataProc with Spark and Hadoop. Manage and schedule data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow) for optimal pipeline performance. Implement effective data migration strategies and design/optimize data layers with services such as BigQuery, BigTable, and Cloud Spanner.
Demonstrated expertise in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem. Proven experience developing and managing batch and real-time data pipelines for Data Warehouses and Datalakes using Google services and open-source technologies (e.g., Apache Airflow, dbt, Spark/Python). Proficiency with Google Cloud Storage, BigTable, BigQuery, DataProc, and DataFlow. Understanding of data platform management through Cloud Scheduler and Cloud Composer. Experience with data layer design using BigQuery, BigTable, and Cloud Spanner. Familiarity with open-source tools like Apache Airflow, dbt, Spark, and Python.
IBM
Information Technology & Services