Lead Data Engineer - Databricks
Salary: £70-£95k (Dependant on Experience)
Location: Can be based anywhere in the UK (Largely remote, must be willing to travel weekly)
Are you an experienced Data Engineer with a passion for leading teams and delivering high-quality solutions?
We are looking for a Lead Data Engineer to join our dynamic consultancy customer.
This role is perfect for someone with a strong technical background in Databricks and Azure Analytics, combined with excellent leadership skills.
Key Responsibilities:
* Databricks Expertise: Implement and optimize workloads in Databricks, with a full understanding of features, access controls, security, and networking. Experience with Unity Catalog is essential.
* Team Leadership: Lead and mentor a team of internal and client engineers, setting high standards in coding, performance, and solution design.
* Solution Architecture: Design and implement scalable data solutions using Azure Data Factory, Storage, Key Vault, Databricks, and/or Fabric Engineering.
* Customer Engagement: Communicate complex technical solutions to non-technical stakeholders with clarity and confidence.
* Technical Community Presence: Stay active in the data engineering community by contributing to events, blogs, or open-source initiatives.
* On-Site Collaboration: Willingness to travel to client sites one day per week (expenses covered).
Required Skills & Experience:
* Expert-level proficiency in Python and Apache Spark.
* Proven experience in Databricks, with a strong understanding of Unity Catalog, ingestion methods, and CI/CD.
* Strong hands-on experience with Azure Data Factory, Key Vault, Storage, networking concepts, and Databricks (Microsoft Fabric a plus!)
* Leadership experience with the ability to shape best practices and develop engineering teams.
* Excellent communication and stakeholder management skills, able to bridge the gap between technical and non-technical audiences.
* Ability to lead and run projects, and engage with key business stakeholders externally and internally.
Join a team that values innovation, collaboration, and professional growth.
Apply now to take the next step in your career