Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and be part of a dynamic team shaping hybrid cloud and AI journeys for leading global companies. As a Data Engineer focused on Google's data platforms, you will architect, build, and maintain robust data engineering solutions within the Google Cloud ecosystem. This role is for innovators who thrive on tackling complex data challenges and driving meaningful client impact.
Your role will involve designing and developing both batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. You will build and manage data engineering solutions with Google Cloud Storage, BigTable, and DataProc with Spark/Hadoop. Ensuring efficient operations, you'll schedule and manage the data platform using Google Cloud Scheduler and Cloud Composer (Airflow). Additionally, you will develop and implement data migration strategies and optimize data layers using services such as BigQuery, BigTable, and Cloud Spanner.
We are seeking professionals with a strong understanding of Google's Cloud ecosystem, including services for data engineering, data migration, and data layer design. Experience with batch and real-time data pipelines for Data Warehouses and Datalakes is essential. Proficiency in Google Cloud Storage, BigTable, BigQuery, DataProc with Spark/Hadoop, and DataFlow with Apache Beam or Python is required. Familiarity with open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala is highly beneficial. Experience with data layer design using BigQuery, BigTable, and Cloud Spanner, along with data migration solutions, is also key.
IBM
IT Consulting