Location: Luton/Hybrid
JOB PURPOSE
With a big investment into Databricks, and with a large amount of interesting data, this is the chance for you to come and be part of an exciting transformation in the way we store, analyse and use data in a fast-paced organisation. You will join as a Senior Data Engineer providing technical leadership to the Data Engineering team. You will work closely with our Data Scientists and business stakeholders to ensure value is delivered through our solutions.
What you'll be doing
1. Develop robust, scalable data pipelines to serve the easyJet analyst and data science community.
2. Highly competent hands-on experience with relevant Data Engineering technologies, such as Databricks, Spark, Spark API, Python, SQL Server, Scala.
3. Work with data scientists, machine learning engineers and DevOps engineers to develop, develop and deploy machine learning models and algorithms aimed at addressing specific business challenges and opportunities.
4. Contribute to development of our in-house data ingestion, data transformation and ML frameworks through new features while supporting the Data Science & Analytics team in deploying pipelines.
5. Coach and mentor the team (including contractors) to improve development standards.
6. Work with Business Analysts to deliver against requirements and realize business benefits.
7. Build a documentation library and data catalogue for developed code/products.
8. Oversight of project deliverables and code quality going into each release.
Requirements of the Role
Key Skills and experience
* Technical Ability: has a high level of current, technical competence in relevant technologies, and be able to independently learn new technologies and techniques as our stack changes.
* Clear communication: can communicate effectively in both written and verbal forms with technical and non-technical audiences alike.
* Complex problem-solving ability: structured, organized, process-driven and outcome-oriented. Able to use historical experiences to help with future innovations.
* Passionate about data: enjoy being hands-on and learning about new technologies, particularly in the data field.
* Self-directed and independent; able to take general guidance and the overarching data strategy and identify practical steps to take.
Technical Skills
* Significant experience designing and building batch and/or streaming data pipelines that involve orchestration and scheduling.
* Hands-on software development experience in Python as well as modern software development and release engineering practices (e.g. TDD, CI/CD).
* Comfortable writing efficient SQL, optimizing queries and debugging issues.
* Significant experience with Apache Spark or any other distributed data programming frameworks (e.g. Flink, Hadoop, Beam).
* Knowledge of data warehousing principles and familiarity with at least 1 cloud-based DWH solution (Redshift, Snowflake, Databricks etc).
* Understanding of Data Management principles (security and data privacy) and how they can be applied to Data Engineering processes/solutions (e.g. access management, data privacy, handling of sensitive data (e.g. GDPR).
* Experienced at building Real-time data pipelines in a commercial production environment with Spark Streaming, Kafka or Beam.
* Experience deploying and monitoring ML models in production by following best practices for MLOps.
* Understanding of ML development workflow and knowledge of when and how to use dedicated hardware.
* Understanding of the challenges faced in the design and development of a streaming data pipeline and the different options for processing unbounded data (pub/sub, message queues, event streaming etc).
* Understanding of the most commonly used Data Science and Machine Learning models, libraries and frameworks.
* Knowledge of the development lifecycle of analytical solutions using visualization tools (e.g. Tableau, PowerBI, ThoughtSpot).
* Hands-on development experience in an airline, e-commerce or retail industry.
* Worked within the AWS cloud ecosystem.
* Experience of building a data transformation framework with dbt.
What you’ll get in return
* Competitive base salary
* Up to 20% bonus
* BAYE, SAYE & Performance share schemes
* Flexible benefits package
* Excellent staff travel benefits
#J-18808-Ljbffr