We are seeking an experienced and skilled Databricks Engineer with data engineering expertise on the Azure tech stack. This is a pivotal role in our digital transformation journey, where you will play a key part in transitioning from legacy systems to a modern data environment leveraging medallion architecture principles.

As a Databricks Engineer, you will design, build, and maintain efficient, scalable data pipelines to ensure smooth data integration and transformation. Additionally, you will set up and manage connections between Databricks and other services. Collaborating closely with data engineers, analysts, architects, third-party vendors, and cross-functional teams, you will contribute directly to our mission of empowering scientists to improve healthcare for all.

Tech Stack & Tools

Essential:

• Databricks

• PySpark/Python

• SQL

Highly Desirable:

• Azure Data Platform: Azure Data Factory, Synapse, Data Lake, Azure SQL DB, OpenAI

Desirable:

• ERP Systems: Microsoft Dynamics 365, Microsoft Dynamics AX, SAP, or similar

• Microsoft Power Platform: PowerApps, Power Automate (Flow)

• Change Data Capture (CDC) tools: Qlik Replicate, Fivetran HVR

• CI/CD: Azure DevOps, GitHub

• AI tools

• APIs for data gathering from external sources

• Power BI

• Other Databases: PostgreSQL, MySQL, Cosmos DB

Key Responsibilities

• Design and implement the core business Unified Data Platform.

• Develop Databricks scripts and scalable data pipelines; enhance frameworks to ensure code and deployment consistency.

• Connect Databricks with other services to integrate disparate data sources.

• Liaise with third-party vendors to scale resources rapidly.

• Oversee the migration of legacy business data to the new platform, retiring older systems.

• Support business-critical reporting during the transformation process.

• Document processes and implement industry best practices.

• Be an active member of a global team.

About You

Experience:

• 3+ years of commercial Databricks experience.

• 7+ years in data warehousing/data lakes.

• Proven senior-level data engineering experience using cloud technology.

Skills:

• Deep understanding of modern data architecture, including medallion architecture, data warehousing concepts, and dimensional modeling.

• Ability to architect solutions and influence senior management with well-reasoned insights.

• Experience working with third-party consultants/vendors and evaluating their work.

Attributes:

• Strong communication skills with the confidence to challenge and defend design decisions.

• Attention to detail and a methodical approach.

• Self-motivated, autonomous, and results-driven.

• Creative problem-solver with a passion for delivering business value.

• Certifications:Relevant industry certifications are a plus.

LI-EMEA LI-REMOTE