Azure Data Engineer
Reading – 1 day a week
£(Apply online only) per day inside IR35
About the Role:
We are seeking a highly skilled Data Engineer with strong expertise in Microsoft Azure, Databricks and SQL to join our data engineering team. This role requires someone who is proactive, solution-oriented, and able to work autonomously within a busy team environment. You will play a crucial role in structuring and integrating core data systems, ensuring high-quality data flows into and out of our Master Data Management (MDM) platform.
This is a fantastic opportunity to join a business that has just secured £50 million in investment and are working on a huge utilities project of national importance. This consultancy is building an elite team of data professionals, already outpacing competitors in the data consultancy space.
📌 Contract or Permanent – Your Choice
* Initial 6-month inside IR35 contract, but huge opportunity to stay for longer as this is a multi-year project with huge growth potential
* Option to go perm if you want long-term stability
Key Responsibilities:
* Azure Data Engineering: Utilize Azure services to manage data pipelines, storage, and processing. Work with Azure Data Lake, Synapse Analytics, and other relevant Azure tools.
* MDM Integration: Ensure data is structured and processed correctly within the MDM platform to produce a golden set of data assets, which serve as an enhanced and high-quality version of source data.
* Data Quality & Analytics: Implement data quality measurement processes, generating analytics to feed into dashboarding and reporting solutions.
* Multi-Source Data Processing: Handle data from four primary sources, managing its ingestion through the data lake, transforming it in Prophecy, applying data quality rules, and enriching it with additional insights.
* Data Pipeline Development: Build and maintain scalable ETL/ELT pipelines to facilitate efficient data flow across the platform.
* Exception Reporting: Develop and maintain exception reporting to monitor inbound data quality and identify discrepancies.
* Collaboration & Autonomy: Work efficiently within a busy team where full support may not always be available, requiring a proactive and forward-thinking approach to problem-solving.
Required Skills & Experience:
* Proven experience as a Data Engineer, with a strong focus on MS Azure, Databricks and SQL.
* Expertise in Azure Data Factory, Synapse, Data Lake, and Prophecy (or similar tools).
* Experience working with Master Data Management (MDM) systems and structuring data for optimal integration.
* Strong knowledge of ETL/ELT pipeline development and data transformation best practices.
* Proficiency in data quality assurance, analytics, and exception handling.
* Ability to work independently, problem-solve proactively, and drive improvements in data processes.
* Understanding of property data assets and their integration from multiple sources is a plus.
Preferred Qualifications:
* Certifications in Microsoft Azure (e.g., Azure Data Engineer Associate)
* Experience with big data technologies and cloud-based data warehousing solutions.
* Familiarity with Power BI or other visualization tools for data analytics.
Data Engineer / Senior Data Engineer / Principal Data Engineer / Data Architect