- Sector: Public sector / Government
- Clearance required: SC clearance
- Contract length: 6 months
- Rate: 400-500/d
- IR35 status: Outside
- Onsite requirement: 2 days a week
- Role title: Data Engineer
Job Description:
1. Build streaming data ingestion pipelines using AWS MSK and batch data ingestion pipelines using APIs.
2. Prepare data for prescriptive & predictive modelling.
3. Experience working with various data formats e.g., parquet, avro, json.
4. Experience in building ETL pipelines using Lambda, Step Functions, and Glue.
5. Experience working with various AWS database services (e.g., RDS, DynamoDB, and Redshift).
6. Technical expertise with various data modelling techniques.
7. Hands-on experience with Python and SQL.
8. Kafka streaming data experience to build new data pipelines.
Supporting Skills:
1. 5 years' experience in an iterative and Agile development environment.
2. Appreciation of Software Development Environments & Languages.
3. Appreciation of Business Analysis Activities.
4. Appreciation of other scripting languages, e.g., JavaScript, Lua, Perl, PHP, Unix.
#J-18808-Ljbffr