Senior Administrator - Python (IN)

HCLTech

8+ yrs India Full Time Remote
HCLTech logo
Posted : today
Actively hiring

Job description

Join our team as a Senior Python & Databricks Developer, focusing on designing and optimizing robust data engineering and analytics solutions within the Databricks environment. This role is key in collaborating with data engineers, data scientists, and stakeholders to construct dependable ETL pipelines and support advanced analytics initiatives.

This position requires extensive experience in building and refining large-scale data processing systems. You will be instrumental in ensuring data integrity, efficiency, and the successful implementation of complex data workflows.

Responsibilities

Key responsibilities include designing, developing, and maintaining ETL/ELT pipelines using Python (PySpark) and Databricks. You will build and optimize Spark jobs for extensive data processing, develop reusable Python code, and implement data transformations. Additionally, you'll manage Databricks notebooks, jobs, and workflows, optimize Spark performance, and work with Delta Lake.

Data integration from various sources like RDBMS, cloud storage, and APIs is crucial. You will also collaborate with cloud platforms (Azure/AWS/GCP), integrate with CI/CD tools, and utilize Git for version control. Active participation in Agile ceremonies and client interaction, including demos and troubleshooting, is expected.

Qualifications

We are seeking candidates with a strong foundation in Python, Databricks, and Apache Spark (PySpark). Essential skills include a solid understanding of data warehousing concepts, experience with Delta Lake, strong SQL abilities, and proficiency in handling large datasets and distributed systems.

Experience with at least one major cloud platform (Azure, AWS, or GCP) and familiarity with CI/CD, Git, and Agile tooling are necessary. Excellent problem-solving, analytical, communication, and documentation skills are vital for success in this client-facing role within an Agile/Scrum framework.

Essential Skills

PythonDatabricksApache SparkPySparkData WarehousingDelta LakeSQLLarge DatasetsDistributed SystemsAzureAWSGCPCI/CDGitAgileJIRAAzure BoardsProblem SolvingAnalytical SkillsCommunicationDocumentationAgile/Scrum

Good to Have

KafkaSpark Structured StreamingData GovernanceData SecurityAccess ControlsMachine LearningDatabricks CertificationsCloud Certifications

Highlights

  • Actively hiring

More Details

RoleSenior Administrator - Python (IN)
Employment TypeFull Time, Remote

About the Company

HCLTech logo

HCLTech

IT Consulting

Senior Administrator - Python (IN) at HCLTech | SkillMX | SkillMX