Senior Data Engineer
Persistent
Persistent
Join our dynamic team as a Senior Data Engineer, focusing on Oracle-based analytical platforms and Apache Spark. You will be instrumental in driving large-scale data modernization, migration, and analytics projects. This role is perfect for an experienced professional passionate about building high-performance data pipelines and optimizing complex data processing workloads within enterprise environments.
Our organization, an AI-led digital engineering partner, collaborates with industry leaders across finance, healthcare, and technology. We are recognized for innovation and sustained growth, offering a challenging yet rewarding environment to leverage cutting-edge technologies and drive significant impact for our clients.
Design, develop, and optimize robust data processing solutions utilizing Apache Spark (PySpark/Scala) and Python. Build and maintain scalable ETL/ELT pipelines for analytical and reporting needs. Enhance query performance, Spark jobs, and end-to-end data pipelines for high-volume batch and near real-time processing.
Implement data integration, replication, and ingestion strategies using tools like Oracle GoldenGate, Kafka, and CDC frameworks. Collaborate with application, data, and analytics teams to create scalable data models and processing frameworks. Lead performance tuning efforts across Oracle, Spark, and distributed data processing systems.
A minimum of 8 years of experience in Data Engineering or a related field is essential. Proficiency in Apache Spark (PySpark and/or Scala) and Python is mandatory. A deep understanding of Oracle databases for OLAP, data warehousing, and analytical tasks is required.
Demonstrated success in tuning and optimizing Spark applications, complex SQL queries, and large-scale ETL/ELT processes. Strong SQL and data modeling skills are crucial. Experience with Oracle GoldenGate, CDC, or enterprise-scale data integration technologies is necessary.
Persistent Systems
IT Consulting