Duties include (but not limited to):
* Provide logical and physical data designs/models.
* Design, develop, and maintain scalable data pipelines using Azure Databricks.
* Ensure efficient data ingestion, transformation, and storage processes.
* Work closely with business stakeholders, architects, analysts, and other stakeholders to understand data requirements.
* Implement and enforce data security measures to protect sensitive information.
* Ensure compliance with data governance policies and regulatory requirements.
* Maintain comprehensive documentation of data pipelines, processes, and architectures.
Qualifications
We are looking for a seasoned Senior Data Engineer. If you have a combination of the following skills and experience, we are keen to chat!
* Strong skills in designing and building ETL/ELT pipelines, including understanding when to use ETL vs ELT.
* DataBricks and Azure proficiency.
* Proficiency in programming languages such as Python, SQL, and PowerShell.
* Familiarity with cloud architecture and best practices, specifically in Azure.
* Experience in data modelling, schema design, and data warehousing concepts.
* OLTP and OLAP modelling experience.
* Securing data using RBAC.
* Understanding of data quality frameworks and data governance principles.
* Some experience mentoring junior data engineers.
* Financial services experience.
* DevOps experience and tooling, specifically Microsoft Azure DevOps.
* Knowledge of Microsoft SQL Server architecture and features.
#J-18808-Ljbffr