Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and build lasting client relationships by shaping hybrid cloud and AI strategies. Leverage IBM technology, Red Hat, and strategic partnerships to drive impactful change. Our collaborative, curiosity-driven culture fosters innovation and supports your long-term career growth.
As a Data Engineer on Google's data platforms, you will design, develop, and maintain robust data engineering solutions. This involves creating batch and real-time data pipelines, implementing data migration strategies, and optimizing data layers using a range of Google Cloud services.
Key tasks include designing and building data pipelines with services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. You will also manage data platforms using Cloud Scheduler and Cloud Composer, and implement efficient data migration solutions. Optimizing data storage and retrieval with services such as BigQuery, BigTable, and Cloud Spanner is also a core part of this role.
We are seeking expertise in Google's Cloud ecosystem for data engineering, including hands-on experience with DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB. Proficiency in developing and managing batch and real-time data pipelines for Data Warehouses and Data Lakes using open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala is essential.
Candidates should demonstrate experience with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). Familiarity with scheduling and managing data platforms via Google Cloud Scheduler and Cloud Composer (Airflow) is required. Experience in designing and implementing data migration solutions is also necessary.
IBM
IT Consulting