We are looking for a Lead Data Engineer to take on a leadership role within our client's Data Engineering team, driving technical excellence and transforming their data ingestion processes. You will be responsible for implementing Data Engineering strategy, working closely with the Head of Data Engineering to define best practices for the team whilst also working closely with the Project team to ensure technical specifications and tasks are clear for all team members.
Responsibilities
* Lead data engineering projects, provide guidance & oversight to junior + mid-level data engineers, and establish best practices to improve the software development life-cycle
* Work closely with the Head of Data Engineering to ensure the data engineering strategy is implemented by the team
* Help define clear testing frameworks for the team to follow for validating/transforming synthetic data
* Define best practices for containerisation of our applications for deployment to multi-node Kubernetes clusters
* Partner closely with AI engineers & our research teams to help in product ideation
* Create and maintain clear documentation on development best-practices, testing protocols & code reviews
* Work closely with the Project team to define “definitions-of-done” & aid in completing technical specifications for User Stories & Epics
* Perform monthly cost reviews on our AWS infrastructure with the DevOps team members to remove obsolete services & suggesting resource optimizations and savings plans where appropriate