Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI journeys for leading companies worldwide. Leverage robust IBM technology, Red Hat, and strategic partnerships to drive meaningful change and accelerate client impact. We foster a culture of curiosity, innovation, and continuous growth, valuing your unique contributions and supporting your long-term career development.
Design, build, and maintain scalable data engineering solutions on Google Cloud. Develop robust batch and real-time data pipelines using services like DataProc, DataFlow, Pub/Sub, and BigQuery. Implement data migration strategies and optimize data layers with BigQuery, Bigtable, and Cloud Spanner. Schedule and manage data platform operations using Cloud Scheduler and Cloud Composer (Airflow).
Proven experience in designing and building data engineering solutions within the Google Cloud ecosystem. Expertise in developing and managing batch and real-time data pipelines using Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala. Proficiency in Google Cloud Storage, Bigtable, BigQuery DataProc, and Google DataFlow with Apache Beam or Python. Familiarity with data layer design using BigQuery, Bigtable, and Cloud Spanner is essential.
IBM
Technology