Job Title: Azure Databricks Data Engineer
Location: UK-Wide Offices - Hybrid (Primarily Remote)
Salary: £70,000 to £100,000 (plus Pension, Benefits, and Bonus)
We’re looking for a skilled Azure Data Engineer to join our team and help build robust, scalable, and secure data platforms. In this role, you’ll work with Databricks, Apache Spark, and Azure technologies to design, develop, and implement data warehouses, lakehouses, and AI/ML models that drive our data-centric initiatives.
Key Responsibilities:
* Build high-performance data pipelines using Databricks and Apache Spark to transform and load data into Azure services.
* Develop secure data solutions by creating data models, applying quality checks, and implementing governance for accuracy and reliability.
* Integrate AI/ML models by building machine learning models in Databricks ML and Azure ML to deliver predictive insights.
* Optimize performance by monitoring and enhancing pipeline performance for scalability and efficiency.
* Collaborate across teams by partnering with business analysts, data scientists, and DevOps engineers to ensure seamless data solutions.
* Stay on the cutting edge by keeping pace with big data trends and best practices to continuously innovate.
Requirements:
* Experience: 3+ years as a Data Engineer or similar.
* Technical skills: Expertise in Databricks, Apache Spark, and Azure (Data Lake Storage, Databricks, Azure Data Factory).
* Programming: Proficient in SQL, Python, or Scala.
* AI/ML skills: Familiarity with machine learning tools and techniques.
* Soft skills: Strong problem-solving, collaboration, and communication skills.
* Passion for data: Enthusiasm for learning and innovation in the data space.
Security Clearance: This position requires Security Check (SC) clearance, including a minimum of 5 years of continuous UK residency.