Lead Data Engineer - AWS, Python, SQL, Kafka - Fully Remote
We are looking for a Lead Data Engineer with expertise in AWS, Python, SQL, and Kafka for a cloud-based data, software development company based in London. Their products allow data transformation into ultra-secure cloud based environments mainly used by large enterprises and government sectors.
Please note: A UK Government Security Check (SC) clearance is required for this role.
Key Responsibilities:
* Develop, test, and document new or updated data pipelines to meet both functional and non-functional business requirements.
* Expand and enhance the data platform’s capabilities to tackle emerging data and analytics challenges.
* Conduct data analysis to propose feasible solutions and improvements for data processing problems.
* Ensure consistency between data models and the broader data architecture, including entity names, relationships, and definitions.
Required Skills & Experience:
* Extensive experience with public cloud platforms, particularly AWS, with hands-on expertise in core services such as EC2, RDS, Lambda, and Athena.
* Proficient in Python and comfortable working with Python-based data engineering tasks.
* Experience configuring and optimising Relational and NoSQL databases.
* Solid understanding of various data structures and their use cases, including their advantages and limitations.
If you're passionate about cloud data engineering and meet the above qualifications, we want to hear from you! Get in touch ASAP for more details.