Join Iris Software, recognized as one of India's Top 25 Best Workplaces in IT, and embark on a career-defining journey. We are a rapidly growing IT services company committed to being a trusted technology partner and a premier destination for top industry professionals. With a global presence across India, the U.S.A., and Canada, we empower enterprise clients with technology-enabled transformations in financial services, healthcare, transportation & logistics, and professional services.

Our expertise spans complex, mission-critical applications utilizing cutting-edge technologies such as Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation. At Iris, your role is a catalyst for growth, with a philosophy of "Build Your Future. Own Your Journey." We foster an environment where your potential is recognized, your contributions matter, and your work drives tangible impact. Benefit from advanced projects, tailored career development, continuous learning, and mentorship to achieve your personal and professional best.

Design and implement scalable data engineering solutions using PySpark and modern distributed data processing frameworks. Develop robust architectures for data ingestion, transformation, and processing, aligning with business objectives. Architect and optimize Snowflake or Delta Lake on Databricks for enterprise-scale data platforms. Lead the creation of high-performance batch and streaming data pipelines, defining streaming standards and integration patterns. Architect workflow orchestration using Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through comprehensive monitoring and scheduling.

Drive data quality, validation, reconciliation, and governance initiatives. Implement Lakehouse architecture principles, data observability, and platform engineering standards for enhanced scalability and reliability. Develop business-focused data products by improving data quality, discoverability, and usability. Leverage AI-assisted engineering to boost productivity, testing, and overall quality. Review data pipeline designs to ensure adherence to standards, troubleshooting complex issues with detailed root cause analysis. Collaborate with cross-functional teams for end-to-end data platform delivery.

We are seeking a seasoned Senior Engineer with mandatory expertise in PySpark, Databricks Workflows, Delta Lake on Databricks, and Amazon Kinesis. A strong foundation in Big Data technologies, particularly PySpark and Apache Spark, is essential.

Proficiency in Data Quality & Validation, Python, and SQL is required. Experience with cloud platforms, specifically AWS services such as SNS, SQS, Kinesis, CloudWatch, S3, Glue, EMR, Redshift, DynamoDB, and RDS, is crucial. Familiarity with ETL & Data Integration, API (SOAP, REST) middleware, and Unix/Linux Shell scripting is also expected.

Key behavioral competencies include strong ownership, effective collaboration, a quality-focused engineering mindset, robust analytical thinking, adaptability to evolving technologies, and clear communication. A high degree of attention to detail, a commitment to continuous improvement, and the ability to balance scalability, performance, reliability, and business priorities are vital for success in this role.