Join Iris Software, recognized among India's Top 25 Best Workplaces in the IT industry. We are a rapidly growing IT services company committed to being our clients' most trusted technology partner and a premier destination for top industry professionals. Our global team of over 4,300 associates across India, the U.S.A., and Canada empowers enterprise clients with technology-driven transformations.

We specialize in complex, mission-critical applications, leveraging cutting-edge technologies in Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation. At Iris, every role serves as a launchpad for your career growth, reflecting our belief that "Build Your Future. Own Your Journey." We cultivate a culture where your potential is recognized, your insights are valued, and your contributions drive tangible impact. Through challenging projects, tailored career development, continuous learning, and dedicated mentorship, we support your personal and professional evolution.

Architect scalable data engineering solutions using PySpark and advanced distributed data processing frameworks. Define robust data ingestion, transformation, and processing architectures aligned with strategic business and analytical goals. Design and optimize enterprise-scale data platforms leveraging Snowflake or Delta Lake on Databricks. Lead the development and implementation of high-performance batch and real-time data pipelines. Architect event-driven data solutions using Apache Kafka or Amazon Kinesis, establishing clear standards and frameworks.

Design and implement workflow orchestration using Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through comprehensive monitoring and scheduling. Drive data quality, validation, reconciliation, and governance initiatives. Develop data engineering solutions based on modern Lakehouse architecture, data observability, and platform engineering principles to enhance scalability and operational visibility. Foster the creation of business-focused data products by improving data quality, discoverability, and usability.

Champion the responsible adoption of AI-assisted engineering tools to boost development productivity, testing, and quality. Conduct thorough reviews of data pipeline designs and implementations, ensuring adherence to engineering, scalability, and performance standards. Address complex data processing, workflow, and streaming platform challenges with detailed root cause analysis. Mentor team members on best practices in PySpark, Snowflake, Delta Lake, Kafka, Kinesis, Airflow, and general data engineering principles. Collaborate effectively with diverse teams and stakeholders to ensure seamless end-to-end data platform delivery.

This role demands expertise in Python, PySpark, Databricks Workflows, Delta Lake on Databricks, and Apache Kafka as mandatory skills. Familiarity with AWS CloudShell and Azure Databricks is also highly beneficial.

Key responsibilities include designing scalable data engineering solutions, defining data architectures for ingestion and transformation, and optimizing solutions for enterprise-scale data platforms. You will lead the implementation of high-performance data pipelines, both batch and streaming, and architect event-driven data solutions. The role involves establishing monitoring and control mechanisms for reliable pipeline execution and driving data quality and governance practices.

We are looking for individuals who can design data engineering solutions based on modern Lakehouse architecture principles and promote responsible AI-assisted engineering. Strong analytical thinking and the ability to troubleshoot complex issues are essential. Mentoring team members and collaborating with stakeholders to support end-to-end platform delivery are also key aspects of this position.