Seeking a skilled Databricks Developer to craft and maintain robust data pipelines and analytics solutions. This role emphasizes strong proficiency in Apache Spark, Python, and SQL, along with cloud data engineering expertise to drive enterprise data integration and analytics.

This position is crucial for developing scalable data solutions that meet complex business needs. You will be instrumental in transforming raw data into actionable insights, ensuring data quality and accessibility for various stakeholders.

Key contributions include designing, developing, and maintaining ETL/ELT pipelines within Databricks.

Develop scalable data processing frameworks using Apache Spark (PySpark) and build/optimize both batch and streaming data pipelines.

Integrate data from diverse sources, including databases, APIs, and cloud storage, while implementing thorough data transformation, cleansing, and validation processes.

Utilize Delta Lake for optimized data storage and create reusable Databricks notebooks, workflows, and jobs. You will also monitor and troubleshoot pipeline issues and collaborate with cross-functional teams to define data requirements.

Essential qualifications include extensive experience with Databricks and a deep understanding of Apache Spark/PySpark.

Proficiency in Python and strong SQL skills are mandatory, alongside practical experience in ETL/ELT development and Delta Lake implementation.

A solid grasp of data warehousing concepts and familiarity with Git for version control are also required.

Experience with cloud platforms like Azure, AWS, or GCP, streaming technologies, orchestration tools such as Airflow, and concepts of data governance and security are highly desirable.