Job Description
Role: Data Engineer
Location: Newcastle-upon-Tyne
We are excited to partner with our client in seeking skilled Data Engineers to join their innovative team. This is a unique opportunity to work on cutting-edge data products and pipelines that deliver high-quality, standardized data for analysts, both internally and across government sectors. These roles are instrumental in providing timely and impactful datasets in response to significant national events.
Role Overview
As a Data Engineer, you will play a key part in developing and maintaining robust data pipelines and ETL processes. You will contribute to high-profile projects, such as creating integrated data sharing platforms and supporting critical national initiatives.
Key Technologies:
* Python, PySpark, SQL
* Cloud platforms: Google Cloud Platform (GCP) or Amazon Web Services (AWS) with Cloudera
Key Responsibilities:
1. Design and build data pipelines and ETL processes.
2. Contribute to data validation packages and automation tools.
3. Work collaboratively within a diverse team while also managing your own workload independently.
4. Maintain high standards of coding, peer review, testing, and documentation.
5. Investigate and resolve data quality issues with innovative solutions.
6. Stay curren...