Join Iris Software, recognized as one of India's Top 25 Best Workplaces in IT, and embark on a career-defining journey. We are a rapidly growing IT services company committed to being a trusted technology partner and a premier destination for top industry professionals. With a global presence across India, the U.S.A., and Canada, we empower enterprise clients in financial services, healthcare, transportation & logistics, and professional services through cutting-edge technology solutions. Our expertise spans Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation, tackling complex, mission-critical applications.

At Iris, every role is a catalyst for growth, guided by our philosophy, “Build Your Future. Own Your Journey.” We cultivate an environment where your potential is recognized, your contributions are valued, and your work drives tangible impact. Through challenging projects, tailored career development, continuous learning, and mentorship, we empower you to achieve personal and professional excellence.

Design and implement robust, scalable data engineering solutions leveraging PySpark and advanced distributed processing frameworks. Define comprehensive data ingestion, transformation, and processing architectures aligned with strategic business and analytical goals. Architect and optimize Snowflake or Delta Lake on Databricks for enterprise-scale data platforms. Lead the development of high-performance batch and real-time data pipelines, including event-driven architectures using Apache Kafka or Amazon Kinesis. Establish data streaming standards, integration patterns, and scalable processing methodologies. Develop workflow orchestration solutions with Apache Airflow or Databricks Workflows, ensuring reliable execution through robust monitoring, scheduling, and operational controls. Drive data quality, validation, reconciliation, and governance initiatives across all data engineering solutions. Implement modern Lakehouse architecture principles, data observability, and platform engineering standards to enhance scalability, reliability, and operational insights. Foster the creation of business-focused data products by improving data quality, discoverability, usability, and documentation, ensuring trusted consumption across analytical platforms. Promote the responsible application of AI-assisted engineering to boost development productivity, testing, and overall quality. Conduct thorough reviews of data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance benchmarks. Provide detailed root cause analysis to troubleshoot complex data processing, workflow, and streaming platform issues. Mentor team members on best practices in PySpark, Snowflake, Delta Lake, Kafka, Kinesis, Airflow, and data engineering principles. Collaborate effectively with cross-functional teams and stakeholders to ensure seamless end-to-end data platform delivery.

Mandatory skills include expertise in Databricks Workflows, Delta Lake on Databricks, Snowflake, PySpark, and Amazon Kinesis. Experience with Apache Kafka and Apache Airflow is also essential for architecting workflow orchestration and event-driven data architectures.

Key competencies encompass Big Data technologies like PySpark and Apache Spark, alongside Data Engineering principles covering Data Quality & Validation. Proficiency in Python, SQL, and various cloud services such as AWS SNS, AWS SQS, and AWS Kinesis is required. Familiarity with Databricks, ETL Concepts, AWS Glue, and database systems including Dynamo DB, Amazon Aurora, Amazon RDS, MS SQL, and PL/SQL is necessary.

Behavioral competencies involve demonstrating strong ownership, effective collaboration with diverse teams and business stakeholders, and a commitment to quality-focused engineering through proactive validation and continuous improvement. Strong analytical thinking is crucial for evaluating complex data engineering and platform challenges.