Key Responsibilities Delivers the designs set by Lead Data engineer and Data Architects. Design and develop Azure Pipelines including data transformation and data cleansing. Document source-to-target mappings. Re-engineer manual data flows to enable scaling and repeatable use. build accessible datasets for distribution and analysis. Key Outputs and Deliverables Development of Azure Pipelines for transforming data. A scalable meta-data driven ingestion and transformation framework. Aligning transformation pipelines and datasets with Purview. Key Stakeholder Relationships Internal Data Engineering Technical Design Lead Data platform Product Owner Data and Digitalisation Programme, Project and delivery managers Data and Digital Insights functional and technical specialists Digital Services technical specialists Ofgem general (BI) and specialised (Data Scientist) end users External Software and Platform Providers Specialist consulting providers Person specification Essential Criteria Experience of working with Azure Data Lakes, data warehousing, and pipelines and storage accounts. Experience in building and managing pipelines including: - building data interfaces to source systems. - combining and transforming data into appropriate storage formats. - engineering data sets for analytics purposes. - developing automated and robust pipelines with error handling. - experience identifying and resolving issues in databases, data processes, data products and services. Proficiency in T-SQL and Python to develop automation. Able to achieve and maintain SC Clearance. Behaviours We'll assess you against these behaviours during the selection process: Changing and Improving Communicating and Influencing Working Together Delivering at Pace Technical skills We'll assess you against these technical skills during the selection process: Please refer to the Candidate Pack and Role Profile attached for full details. Benefits Alongside your salary of £47,895, OFGEM contributes £13,875 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides. Ofgem can offer you a comprehensive and competitive benefits package which includes; 30 days annual leave after 2 years; Excellent training and development opportunities; The opportunity to join the generous Civil Service pension which also includes a valuable range of benefits; Hybrid working, flexible working hours and family friendly policies. Plus lots of other benefits including clean and bright offices based centrally, engaged networks and teams and an opportunity to contribute to our ambitious and important targets of establishing a Net Zero energy system by 2050. This exciting blend of professional challenge and personal reward identifies career opportunities at Ofgem as something to get excited about. Things you need to know Selection process details This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills. When you press the 'Apply now' button, you will be asked to complete personal details (not seen by the sift panel), your career history and qualifications. You will then be asked to provide a 1250 word 'personal statement' evidencing how you meet the essential and desirable skills and capabilities listed in the role profile. Please ensure you demonstrate clearly, within your supporting statement, how you meet each of the essential and desirable skills and capabilities. The Civil Service values honesty and integrity and expect all candidates to abide by these principles. Ofgem take any incidences of cheating very seriously. Please ensure all examples provided are of your own experience. Any instances of plagiarism or other forms of cheating will be investigated and, if proven, the relevant applications will be withdrawn from the process. Feedback will only be provided if you attend an interview or assessment. Security Successful candidates must undergo a criminal record check. Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window). See our vetting charter (opens in a new window). People working with government assets must complete baseline personnel security standard (opens in new window) checks. Nationality requirements This job is broadly open to the following groups: UK nationals nationals of the Republic of Ireland nationals of Commonwealth countries who have the right to work in the UK nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window) nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS) individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020 Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service Further information on nationality requirements (opens in a new window) Working for the Civil Service The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants. We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window). The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria. The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy. Diversity and Inclusion The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window) .