Senior Data Engineer
Salary- Up to £90k
Location- London (Hybrid)
This is an exciting opportunity to work with cutting-edge cloud-native data platforms (AWS, Azure, Databricks) and technologies like Spark, Delta, Kafka, and Terraform. You’ll collaborate with cross-functional teams, mentor junior engineers, and drive best practices in data engineering.
Key Responsibilities:
* Design & Build Data Pipelines: Develop and optimize batch and real-time data pipelines using Python, SQL, Spark, and Delta.
* Data Transformation & Processing: Implement ELT workflows to ingest, enrich, and publish data for analytics and AI/ML.
* Security & Compliance: Ensure data privacy, governance, and regulatory compliance.
* Performance Optimization: Tune queries and pipelines for cost-efficiency and speed.
* Monitoring & Observability: Integrate real-time monitoring to detect and resolve issues proactively.
* CI/CD & DevOps: Implement Terraform, GitHub, and DevOps pipelines for automation and deployment.
* Collaboration & Mentorship: Work closely with product owners, analysts, and data scientists, while mentoring junior engineers.
* Research & Innovation: Stay ahead of the curve with the latest data technologies and best practices.
What We’re Looking For:
* Strong experience in Python & SQL for Databricks and Spark (including Delta, Delta Live Tables, PyTest, Great Expectations).
* Expertise in batch and streaming data processing using Kafka, AWS Kinesis, or Azure Stream Analytics.
* Hands-on experience in cloud-native data platforms (AWS, Azure, or Databricks).
* Knowledge of data governance, privacy laws (e.g., GDPR), and security best practices.
* Experience with DevOps for data engineering – Terraform, GitHub, CI/CD.
* Agile mindset – experience working in Scrum/Kanban teams.
* Ability to mentor junior engineers and conduct code reviews (PR reviews).
Please reach out to- daniel.wexler@xcede.com for more information!
Unfortunately, they do not offer sponsorship