Job Description
Data Engineer (Python Spark SQL)
We are seeking a highly skilled Data Engineer to join our team in Newcastle. This is an exciting opportunity to progress your career at a start-up Investment Management firm with secure backing, an established Hedge Fund client as a partner and massive growth potential.
About the Role:
The successful candidate will be instrumental in transitioning our core data platform to Azure, managing 3rd party data ingestion and integration. You will collaborate and be supported by a small team, seeking ways to improve the data quality and introduce event streaming.
Responsibilities:
* Design, develop and maintain large-scale data pipelines using Python, Apache Spark and SQL.
* Collaborate with cross-functional teams to integrate data into Azure.
* Manage and optimize data ingestion and processing workflows.
* Implement event-driven architecture using Apache Kafka and Spark Streaming.
* Maintain high-quality documentation of data engineering processes and architectures.
Requirements:
* 2:1 or higher degree in Computer Science or similar technical/scientific discipline from a top-tier university.
* Commercial experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas.
* Good understanding of modern data engineering best practices.
* Azure and Databricks experience is a plus.
What We Offer:
We offer a competitive salary up to £70,000, significant bonus, and benefits package. Our team is passionate about innovation and continuous learning, and we believe in a diverse workplace that allows people to play to their strengths and continually learn. If you are excited to join a start-up in a role that you can shape and influence, apply now to find out more about this Data Engineer (Python Spark SQL) opportunity.