Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly expanding IT services company focused on enabling enterprise clients through technology-driven transformations.

Our vision is to be the most trusted technology partner, offering opportunities for professionals to realize their full potential. With a global presence across India, the U.S.A., and Canada, we engage in complex, mission-critical applications utilizing cutting-edge technologies in areas like Data & Analytics, Cloud, and DevOps.

Design and implement scalable data engineering solutions, focusing on PySpark and distributed data processing frameworks. Define robust data ingestion, transformation, and processing architectures aligned with business goals.

Architect and optimize enterprise-scale data platforms using Snowflake or Delta Lake on Databricks. Lead the development of high-performance batch and streaming data pipelines, incorporating event-driven architectures with technologies like Amazon Kinesis.

Establish standards for data streaming, integration, and processing patterns. Develop workflow orchestration solutions with Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through comprehensive monitoring and scheduling.

Drive data quality, validation, and governance practices. Implement modern Lakehouse architecture principles and data observability for enhanced scalability and operational visibility.

Develop business-focused data products by improving data quality, discoverability, and usability. Leverage AI-assisted engineering to boost productivity and quality in development and testing.

Review data pipeline designs and implementations against engineering, scalability, and performance standards. Troubleshoot complex issues through detailed root cause analysis and mentor team members on best practices.

This senior engineering role requires expertise in designing and implementing scalable data engineering solutions.

Key responsibilities include architecting data pipelines using PySpark, Snowflake, Delta Lake on Databricks, and integrating event-driven architectures with Amazon Kinesis or Apache Kafka. Experience with workflow orchestration tools like Apache Airflow or Databricks Workflows is essential.

You will also be responsible for driving data quality, validation, and governance, adhering to Lakehouse architecture principles and observability practices. A strong understanding of Big Data technologies, particularly PySpark and Apache Spark, is crucial.

Proficiency in cloud platforms like AWS (Kinesis, SNS, SQS) and database technologies (Snowflake, Databricks, MS SQL, DynamoDB, Aurora, RDS, PL/SQL, SQL) is expected.

Behavioral competencies include strong ownership, effective collaboration, a quality-focused engineering mindset, and robust analytical thinking.