Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and collaborate with leading global companies to shape their hybrid cloud and AI strategies. Leverage IBM technology, Red Hat, and strategic partnerships to drive impactful change.
This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem. You will work with cutting-edge Google services to create efficient batch and real-time data pipelines, manage data migration, and optimize data layers.
Curiosity is key here. We encourage challenging the status quo and exploring innovative solutions. Our culture supports your career growth and values your unique contributions, fostering a positive and empathetic environment.
- Design and develop batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. - Build and maintain data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). - Manage data platform operations through Google Cloud Scheduler and Cloud Composer (Airflow). - Implement data migration solutions for seamless system data transfers. - Design and optimize data layers using BigQuery, BigTable, and Cloud Spanner.
- Proven experience in designing, building, and maintaining data engineering solutions on Google Cloud. - Expertise in Google services such as DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB. - Experience developing and managing batch and real-time data pipelines for Data Warehouse and Datalake using Google services and open-source technologies (Apache Airflow, dbt, Spark/Python, Spark/Scala). - Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). - Knowledge of data platform management using Google Cloud Scheduler and Cloud Composer (Airflow). - Experience with data layer design using BigQuery, BigTable, and Cloud Spanner. - Familiarity with open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala. - Experience implementing data migration solutions using Google services.
IBM
IT Consulting