Salary: 104,000 - 130,000 GBP per year Requirements:
* I require a strong understanding of data concepts including data types, data structures, and schemas (both JSON and Spark), along with schema management.
* I expect proficiency in complex JSON manipulation.
* Experience working with data pipelines using custom Python/PySpark frameworks is essential.
* I need a solid understanding of the four core data categories: Reference, Master, Transactional, and Freeform, particularly in managing and handling Reference Data.
* Knowledge of data security principles, including data ownership and access controls (row and column level) and GDPR compliance is mandatory, along with experience in handling sensitive datasets.
* Strong problem-solving and analytical skills are necessary, demonstrating the ability to intuitively work through problems rather than simply following instructions.
* Experience in a support role is beneficial, particularly with skills in incident triage and handling, including Service Level Agreements (SLAs).
* Fundamental knowledge of Linux system administration is required, including SSH keys and configuration, Bash CLI and scripting, and environment variables.
* Proficiency in using browser-based IDEs such as Jupyter Notebooks and RStudio is needed.
* Experience in a dynamic Agile environment (SAFE, scrum, sprints, JIRA) is expected.
Responsibilities:
* I will be looking for someone who is knowledgeable in several languages and frameworks including JSON, YAML, and Python (beyond basic scripts; experience with Pydantic is a bonus).
* SQL, PySpark, Delta Lake, and Bash (both CLI and scripting) skills are essential.
* Proficiency with Git and Markdown is expected.
* Experience with Scala is a bonus, but not mandatory.
* Familiarity with Azure SQL Server as a HIVE Metastore is an added advantage.
Technologies:
* Azure
* Bash
* Databricks
* Git
* Hive
* ICT
* Support
* JIRA
* JSON
* Jupyter
* Linux
* Python
* PySpark
* SQL
* Scala
* Security
* Spark
* Cloud
More:
I work with cutting-edge technologies such as Azure Databricks, Apache Spark, Delta Tables, and data processing with Python. Experience with PowerBI for integration and data ingestion is also valuable. LA International is a HMG approved ICT Recruitment and Project Solutions Consultancy, operating globally with our primary base in the UK. We welcome applications from all sections of the community and individuals with diverse experiences and backgrounds. We are proud to be an award-winning organization, recognized for excellence in IT recruitment and having received the prestigious Queens Award for Enterprise: International Trade for two consecutive periods.