Lead Data Engineer required by our market leading, award winning, retail sector client based in Avonmouth. Our client have recently migrated across to Microsoft Dynamics Business Central and require a Lead Data Engineer to lead the development, maintenance, support, and integration across an increasing array of data sources into an Azure-hosted SQL data warehouse and the Power BI platform.
This is a hybrid role with 2-3 days a week onsite but with flexibility.
The Lead Data Engineer will oversee the technical aspects of Reporting & BI solutions within the Technology team, providing robust reporting capabilities to support Commercial, Operational, and Finance functions.
Main Responsibilities
* Analyse, optimise, and refactor existing stored procedures, ETL jobs, and data workflows to improve performance and reduce load times on the current data warehouse.
* Clean up and enhance the efficiency of the current Power BI Premium (P1) instance, ensuring optimal resource allocation and utilisation as data volumes and complexity grow.
* Review and maintain the scheduling of data jobs to ensure timeliness and reliability, improving operational processes to meet evolving business needs.
* Implement data quality checks, governance, and performance monitoring for both ETL processes and reporting outputs.
* Lead and actively execute the long-term migration of the on-premises data warehouse to Microsoft Fabric, including hands-on development and implementation of a robust strategy for transitioning datasets, ETL pipelines, and reporting solutions to the cloud.
* Build scalable ETL processes using Azure Data Factory, Azure Synapse, and other Fabric-native tools, ensuring a smooth integration between the on-prem and future cloud environments.
* Design and implement Azure-based data architecture, leveraging tools like Azure Data Lake, Azure Synapse Analytics, and Fabric’s data capabilities to future-proof the platform.
* Utilise Azure tools such as Synapse Analytics, Logic Apps, and Azure Functions to streamline data processing and orchestration across the business.
* Empower stakeholders to build their own reports and dashboards by enhancing data accessibility and developing self-service capabilities.
Required Background
* Extensive experience in maintaining and optimising on-premises SQL Server data warehouses, focusing on stored procedures, indexing, partitioning, and load performance.
* Deep understanding of Azure cloud tools such as Azure Synapse Analytics, Azure Data Lake, Azure Data Factory, and Logic Apps
* Hands-on experience with Microsoft Fabric or similar modern cloud platforms, with a strong focus on data migration strategies and ETL development
* Proficiency in writing advanced SQL queries, stored procedures, and Python scripts for data manipulation, pipeline development, and automation
* Experience with ETL tools (both on-prem like SSIS, and cloud-based like Azure Data Factory) for creating, optimising, and maintaining data pipelines
* Skilled in Power BI Premium, including data modelling, DAX, performance tuning, and advanced reporting features like composite models and incremental refresh
* Strong understanding of data governance, security best practices, and compliance requirements (GDPR, etc.)
* Experience working with Agile methodologies
* Excellent communication skills, capable of collaborating with both technical and non-technical stakeholders to translate business needs into effective data solutions.
The ‘Nice to Haves’
* Experience with Microsoft Dynamics 365 Business Central
* Experience of defining and operating within data management best practice frameworks
* Experience working in a Retail environment
This fantastic opportunity comes with a competitive starting salary which is accompanied by the following benefits: 7-10% annual bonus, Health Cash Plan, Life Assurance, 25 days annual leave and a generous staff discount to name but a few.
#J-18808-Ljbffr