Senior Engineer - Data Engineering
Altimetrik
Altimetrik
Join our team as a Senior Data Engineer, playing a pivotal role in shaping our data engineering strategy. You will focus on designing, building, and maintaining robust data infrastructure primarily on Google Cloud Platform (GCP).
Your expertise in GCP services like Data Engineering Pipelines, BigQuery, and DataFlow will be crucial in developing and refining our data pipelines. The goal is to ensure these systems are scalable, reliable, and can adapt to our organization's evolving data needs.
You will lead data setup within GCP, prioritizing data integrity and designing for scalability and performance.
Develop and enhance data pipelines using GCP services (BigQuery, DataFlow, Batch processing), incorporating best practices and ensuring maintainability. Experience with automated data lineage is essential.
Implement and uphold data governance and security standards within GCP, ensuring regulatory compliance.
Integrate data pipelines into our CI/CD processes for streamlined deployment.
Continuously monitor and optimize pipeline and platform performance for speed and reliability.
Troubleshoot and resolve data pipeline and platform issues promptly to maintain continuous data flow.
Generate reports on data pipeline performance and quality, analyzing trends for improvement.
We seek an experienced professional with a minimum of 4 years in development and 6+ years in IT, demonstrating practical experience in at least two programming languages or advanced proficiency in one.
A Bachelor's Degree is required.
Demonstrated experience with Google Cloud Platform, specifically BigQuery, DataFlow, Dataproc, Data Fusion, Terraform, Tekton, Cloud SQL, Airflow, PostgreSQL, Airflow PySpark, Python, and APIs is essential.
Familiarity with Data Engineering, DevOps, and CI/CD principles is expected.
Preferred qualifications include experience with BigQuery, Airflow, GCP, Astronomer, and any ETL technology.
Altimetrik
IT Consulting