Data Engineer-Data Platforms-Google
IBM
IBM
Embark on a career in IBM Consulting, where you'll build lasting client relationships and collaborate globally. Drive transformation with leading companies on their hybrid cloud and AI journeys, leveraging IBM technology, Red Hat, and strategic partnerships. Your curiosity will be key to challenging norms, exploring new ideas, and creating innovative solutions.
This role focuses on designing, building, and maintaining robust data engineering solutions within Google's cloud ecosystem. You will be instrumental in developing efficient batch and real-time data pipelines, facilitating data migration, and architecting optimal data layers.
Key responsibilities include architecting and developing batch and real-time data pipelines for data warehouses and data lakes using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.
You will also develop comprehensive data engineering solutions, manage data platforms using Google Cloud Scheduler and Cloud Composer (Airflow), and implement effective data migration strategies.
Furthermore, the role involves designing and optimizing the data layer using services such as BigQuery, BigTable, and Cloud Spanner to ensure efficient data storage and retrieval.
We seek individuals with hands-on experience in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem. Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python) is essential.
Experience in developing and managing batch/real-time data pipelines using open-source technologies like Apache Airflow, dbt, Spark (Python/Scala) is highly valued.
Familiarity with data platform management via Google Cloud Scheduler and Cloud Composer, along with expertise in data layer design using BigQuery, BigTable, and Cloud Spanner, is expected. Experience with data migration solutions using Google services is also a plus.
IBM
Data Analytics