Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly growing IT services company committed to being a trusted technology partner and a premier destination for top industry professionals. With over 4,300 associates globally, we specialize in transforming enterprises across financial services, healthcare, transportation & logistics, and professional services using cutting-edge technologies.

Our expertise spans Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation, tackling complex, mission-critical applications. At Iris, we believe in empowering our employees, offering a launchpad for growth where you can "Build Your Future. Own Your Journey." We foster an environment where your potential is valued, your voice is heard, and your work makes a tangible impact. Benefit from cutting-edge projects, personalized career development, continuous learning, and mentorship.

Discover what it's like to work at Iris by watching our "inside look" video, showcasing our people, passion, and possibilities.

As a Senior ETL Data Engineer, you will be instrumental in designing and implementing scalable data engineering solutions using PySpark and modern distributed data processing frameworks. You will define robust data ingestion, transformation, and processing architectures tailored to business and analytical needs.

Key responsibilities include designing and optimizing Snowflake or Delta Lake on Databricks for enterprise-scale data platforms, leading the implementation of high-performance batch and streaming data pipelines, and establishing data streaming standards. You will architect workflow orchestration using Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through effective monitoring, scheduling, and operational controls.

You will drive data quality, validation, reconciliation, and governance practices. Designing solutions based on Lakehouse architecture principles, data observability, and platform engineering standards will be crucial for enhancing scalability, reliability, and operational visibility. Champion the development of business-focused data products by improving data quality, discoverability, usability, documentation, and trusted data consumption. Promote responsible AI-assisted engineering to boost development productivity and quality.

Your role involves reviewing data pipeline designs and implementations to ensure adherence to standards, troubleshooting complex issues via root cause analysis, and collaborating with cross-functional teams for end-to-end data platform delivery. Behavioral competencies such as strong ownership, effective collaboration, quality focus, analytical thinking, adaptability, clear communication, attention to detail, and a commitment to continuous improvement are essential.

We are seeking a Senior ETL Data Engineer with mandatory expertise in PySpark, Databricks Workflows, Delta Lake on Databricks, and Amazon Kinesis. Proficiency in Big Data processing, particularly with Apache Spark and Python, is essential.

Strong database programming skills including SQL are required. Cloud expertise, specifically with AWS services such as SNS, SQS, Kinesis, CloudWatch, S3, IAM, Secrets Manager, KMS, API Gateway, Glue, EMR, Redshift, DynamoDB, Aurora, RDS, and EC2, is necessary. Experience with Data Engineering concepts like ETL & Data Integration, and Data Quality & Validation is a must.

Familiarity with API middleware (SOAP, REST) and Unix/Linux Shell scripting is also required. Behavioral competencies such as strong ownership, effective collaboration, quality-focused engineering, analytical thinking, adaptability, clear communication, attention to detail, continuous improvement, and balancing priorities are vital for success in this role.