Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and build transformative client relationships by shaping hybrid cloud and AI journeys. Collaborate globally with leading companies and leverage IBM technology, Red Hat, and strategic partnerships to drive significant change. Embrace a culture of curiosity and innovation, where challenging the status quo leads to impactful solutions and accelerated client success. This role focuses on developing a career in data engineering within Google's cloud ecosystem.
Design and construct robust data engineering solutions on Google Cloud. Build and maintain batch and real-time data pipelines using services like DataProc, DataFlow, PubSub, BigQuery, and Big Table. Develop data engineering solutions with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). Manage data platform operations through Google Cloud Scheduler and Cloud Composer (Airflow). Implement and manage data migration strategies for seamless system transfers. Optimize data layers using BigQuery, Big Table, and Cloud Spanner for efficient storage and retrieval.
Demonstrated experience in designing, building, and maintaining data engineering solutions within Google's Cloud ecosystem. Proficiency in developing and managing batch and real-time data pipelines for Data Warehouses and Data Lakes using Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala. Strong experience with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). Knowledge of scheduling and managing data platforms using Google Cloud Scheduler and Cloud Composer (Airflow). Understanding of data layer design utilizing BigQuery, Big Table, and Cloud Spanner. Experience in developing and implementing data migration solutions using Google services. Expertise with Cloud Composer (Airflow) for data platform management. Familiarity with open-source technologies such as dbt, Spark/Python, or Spark/Scala.
IBM
IT Consulting