ETL Data - Senior Engineer
Iris Software
Iris Software
Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly growing IT services company committed to being a trusted technology partner and a platform for top talent to realize their full potential.
Our vision extends to transforming enterprises across financial services, healthcare, transportation & logistics, and professional services. We specialize in complex, mission-critical applications utilizing cutting-edge technologies in Application & Product Engineering, Data & Analytics, Cloud, DevOps, and more.
Design and implement scalable data engineering solutions using PySpark and distributed frameworks. Define data ingestion, transformation, and processing architectures. Optimize Snowflake or Delta Lake on Databricks for enterprise data platforms. Lead the development of high-performance batch and streaming data pipelines.
Architect workflow orchestration using Apache Airflow or Databricks Workflows. Establish monitoring and operational controls for reliable pipeline execution. Drive data quality, validation, and governance practices. Design solutions based on Lakehouse architecture principles, data observability, and platform engineering standards.
Promote the responsible use of AI-assisted engineering to enhance productivity and quality. Review data pipeline designs for adherence to standards. Troubleshoot complex data processing and streaming issues. Collaborate with cross-functional teams for end-to-end data platform delivery.
Possess strong ownership and drive for data engineering excellence. Collaborate effectively with diverse teams and stakeholders. Champion quality-focused engineering through proactive validation and optimization. Apply analytical thinking to complex data challenges.
Demonstrate adaptability to evolving technologies and business needs. Communicate delivery status, risks, and dependencies clearly. Maintain meticulous attention to detail in architecture, design, and implementation. Foster continuous improvement in data engineering practices.
Balance scalability, performance, reliability, and business priorities. Promote innovation by adopting modern data engineering and platform engineering principles. Experience with PySpark, Databricks Workflows, Delta Lake on Databricks, and Amazon Kinesis is mandatory. Proficiency in Big Data, Apache Spark, Python, SQL, and various AWS services is required.
Iris Software
Data Analytics