Data Platform Engineer
£450 - £500/day (Outside IR35)
3 months (initially)
Scotland (remote)
AI Connect are looking for an experienced Data Platform Engineer to work with an exciting AI scale up in Scotland. The client is at the beginning of their data strategy and are looking for a strong engineer with understanding of data platforms, automated pipelines, architecture.
Role responsibilities/ experience required:
* Design and deploy data pipeline architecture to refine and automate existing data collection & processing workflows.
* Understand, document, and validate current data processing methods with input from our current stakeholders.
* Define roadmap for knowledge transfer to permanent team members and data platform architecture and implementation roadmap.
* Document existing manual data workflows identifying opportunities for improvement and highlighting inefficiencies.
* Begin automation of data collection, processing, and storage.
* Initiate the creation of a structured data repository with associated data management methods for research & commercial data
* High-level architecture and implementation roadmap for data platform.
* Optimising, and maintaining data architectures, data storage solutions, and data pipelines to collect, clean, and organise raw data from multiple sources.
* Proficient in SQL and experience with NoSQL databases (e.g., MongoDB, Cassandra).
* Experience with ETL tools (e.g., Apache Airflow, Talend, or similar).
* Familiarity with containerization technologies (e.g., Docker, Kubernetes).
* Strong understanding of data modelling, data governance, and data security principles
If you have the required skills/ experience, then please apply for more details.