Data Engineer-Data Platforms-Google
IBM
IBM
Embark on a career in IBM Consulting, where long-term client relationships and global collaboration are paramount. You'll partner with leading companies across diverse industries, guiding their hybrid cloud and AI transformations. Leveraging robust IBM technology, Red Hat, and support from strategic partners, you'll possess the tools to drive significant change and amplify client impact. Curiosity is the engine of success at IBM Consulting; we encourage challenging the status quo, exploring novel ideas, and crafting innovative solutions that deliver tangible results. Our growth-oriented and empathetic culture is dedicated to your long-term career development, valuing your distinct skills and experiences.
As a Data Engineer specializing in Google's data platforms, you will architect, construct, and maintain data engineering solutions within Google's Cloud ecosystem. This involves leveraging various Google services for both batch and real-time data pipelines, data migration, and designing efficient data layers.
Key contributions include: - Designing and developing batch and real-time data pipelines for Data Warehouses and Datalakes using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. - Building and maintaining data engineering solutions with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). - Managing the data platform, including scheduling data pipelines, via Google Cloud Scheduler and Cloud Composer (Airflow) for optimal operations. - Implementing robust data migration solutions using Google services to ensure seamless data transfers. - Optimizing the data layer with services such as BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval.
Demonstrated experience in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, utilizing services such as Google DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.
Proficiency in developing and managing batch and real-time data pipelines for Data Warehouses and Datalakes using Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.
Experience working with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python) to construct and maintain data engineering solutions.
Knowledge of scheduling and managing data platforms using Google Cloud Scheduler and Cloud Composer (Airflow) for efficient data pipeline operations.
Understanding of data layer design using Google services such as BigQuery, BigTable, and Cloud Spanner for effective data storage and retrieval.
Experience developing and implementing data migration solutions using Google services for seamless data transfer between systems.
IBM
IT Consulting