Data Engineer Salary: £75K My client is an award-winning search intelligence platform with the unique combination of patented, AI-driven technology, and a supportive team of paid search experts, to help brands, marketers, and agencies dominate their competitive landscapes. Responsibilities: Understand distributed technologies and the best practices around them Build and maintain services/features/libraries that serve as a definitive example for new engineers Design and write effective complex Spark jobs (data processes, aggregations, pipeline) Design and write complex asynchronous, highly parallel low latency APIs and processes Work as part of an Agile team to maintain, improve, and monitor data collection processes using Scala and Java Apply industry practices such as TDD and SOLID, and also Engineering Practices Understand and be able to apply data structures and algorithms Understand the platform's data architecture and use appropriate design patterns Design and implement databases for relational and non-relational storage technologies. Support our Data Science team to help deliver their machine learning models in to production environments Be empathetic towards team members and customers Preferred Qualifications: Bachelor's degree in Computer Science, similar technical field of study or equivalent practical experience. Commercial experience developing Spark Jobs using Scala Commercial experience using Java and Scala (Python is nice to have) Experience in data processing using traditional and distributed systems (Hadoop, Spark, AWS - S3) Experience designing data models and data warehouses. Strong understanding and application of data structures and algorithms in building efficient solutions. Experience in SQL, NoSQL database management systems (PostgreSQL and Cassandra) Commercial experience using messaging technologies (RabbitMQ, Kafka) Experience using orchestration software (Chef, Puppet, Ansible, Salt) Confident with building complex ETL workflows (Luigi, Airflow) Good knowledge of working cloud technologies (AWS, GCP) Good knowledge using monitoring software (ELK stack) Motivated problem-solving skills, ability to bring ideas forward and adapt solutions to complex challenges Technologies used: Languages: Java, Scala, JavaScript (React, Backbone), SQL and scripting using Bash and Python Frameworks: DropWizard, React, Akka and Play Framework (Scala) Databases: PostgreSQL, AWS(S3), Redshift, Redis, MongoDB, Cassandra Technologies: RabbitMQ (messaging), Quartz scheduling, Docker and Kubernetes, Maven CI/CD: TeamCity, Jenkins Source Control: Git (GitHub) Other Tools: IntelliJ IDEA, Jira, Grafana Here’s what's on offer: Startup Engineering culture Strong Culture: Quarterly surveys show a 93% average approval rating for company culture, with an overall eNPS of 27. Trust-Based Vacation: Take as much time off as you need, when you need it. Remote-First: 52% work fully remote, with others choosing Hybrid Work at our 3 hubs (London, Austin, Sydney). Flexible Work: Work how and where you do your best, with full autonomy over your day. Monthly Training Days Private Medical Insurance: Fully covered health care. Mental Health Support: Employee Assistance Program offering 24/7 confidential counselling, with access to ‘healthier living’ services such as ‘HeadSpace’. Family Care Package: Up to 6 months fully paid maternity leave, and 2 months of paternity leave. Pension: Competitive UK pension schemes, available from day one. Home Office Stipend: $200 for your ideal remote setup. Swag Welcome Gift: $70 credit to grab some merch. Birthday Day Off: Celebrate your special day. Regular Socials: Summer & Christmas parties, annual and quarterly offsites, monthly meet-ups. Volunteer Days: Make a difference with paid volunteer time each quarter.