Senior Data Engineer
Persistent
Persistent
Join Persistent as a Senior Data Engineer II and drive innovation in our AI-led digital engineering initiatives. This role focuses on designing, developing, and maintaining robust enterprise data pipelines critical for analytics-driven products. You'll collaborate with a global team to deliver high-quality data solutions that power business experimentation and advanced analytics, making a tangible impact on industry leaders worldwide.
Key responsibilities include architecting and implementing scalable ETL and data integration pipelines, and managing enterprise data platforms for a global clientele across various sectors.
You will build and optimize data processing workflows using Spark and Databricks, developing reusable, efficient code for large-scale data volumes. This involves designing and maintaining ETL processes for ingestion, transformation, validation, and delivery, alongside writing and optimizing complex SQL queries.
Further duties encompass partnering with analytics teams to translate business needs into technical solutions, performing data validation and quality assurance, and contributing to metadata management and data governance efforts. Automation of repetitive tasks, participation in code and design reviews, and meticulous documentation are also vital components of this role.
We are seeking candidates with 6 to 12 years of extensive experience in Data Engineering, ETL Development, Data Integration, or Analytics Engineering. A strong command of Apache Spark and Databricks for large-scale data processing is essential.
Proficiency in SQL, including writing, optimizing, and troubleshooting complex queries, is required. Experience with Microsoft SQL Server and relational database technologies, coupled with a solid understanding of data warehousing concepts and data modeling principles, is crucial.
Ideal candidates will possess a knack for handling large datasets, optimizing performance, and implementing data validation techniques. Familiarity with data governance, version control systems like Git, and working within Agile/Scrum environments are highly valued. Excellent analytical, problem-solving, and communication skills are necessary to effectively convey technical concepts to diverse stakeholders.
Persistent Systems
IT Consulting