Lead Data Engineer

Iris Software

8+ yrs Noida Full Time Work from office
Iris Software logo
Posted : 1 week ago
Actively hiring

Job description

Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly expanding IT services company committed to being a trusted technology partner and a platform for professionals to achieve their full potential. Our global team collaborates across India, the U.S.A., and Canada, driving technology-enabled transformations for enterprise clients in financial services, healthcare, transportation, and professional services.

We specialize in complex, mission-critical applications leveraging cutting-edge technologies such as Data & Analytics, Cloud, DevOps, Data & MLOps, and Business Automation. At Iris, we believe in empowering our employees with ownership of their career journeys, fostering a culture of growth, innovation, and impactful work.

Responsibilities

Lead the development and maintenance of robust ETL pipelines, primarily on AWS cloud infrastructure. Oversee data pipelines, data lakes, data warehouses, and ETL/ELT processes, ensuring high standards of data quality and system health. Drive continuous improvements through diligent monitoring and proactive issue resolution.

Execute hands-on development using Python and advanced SQL, with a strong understanding of data modeling concepts, dimensions, and facts. Leverage the Spark ecosystem and PySpark programming for efficient data processing. Convert existing SSIS workflows to scalable AWS ETL pipelines, utilizing services like Lambda, Glue, EMR, EC2, and DMS.

Collaborate closely with business stakeholders, developers, and administrators to accurately capture data requirements. Understand and implement data lakehouse architectures. Document migration procedures, processes, and best practices to ensure knowledge transfer and maintainability.

Qualifications

A Bachelor’s degree in Computer Science, Information Technology, or a related field is required, coupled with over 8 years of experience in data migration or similar roles. Proficiency in SSIS and proven experience building data migration pipelines on AWS are essential.

Demonstrate strong analytical and problem-solving abilities, with a solid understanding of data modeling and schema design principles. Possess excellent communication and collaboration skills, enabling effective teamwork and independent work. Familiarity with data governance, data quality, and CI/CD processes is expected.

Essential Skills

PythonAdvanced SQLSparkPySparkAWSData Pipeline AutomationData GovernanceData QualityAirflowStep FunctionsGitSSISData ModelingData LakehouseCI/CDTypeScript

Good to Have

KafkaKinesisAIPower BIPostgreSQLDBTTerraformAWS Data ZoneAgile

Highlights

  • Actively hiring

More Details

RoleLead Data Engineer
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Work from office

About the Company

Iris Software logo

Iris Software

Information Technology & Services

Lead Data Engineer at Iris Software | SkillMX | SkillMX