Design, develop and implement AWS Serverless Solution for Data Migration from Cloud Data Lake to AWS.
Design and implement the deployment pipeline flow using Azure Devops.
Perform development activities relating to data architectures and ETL processes.
Interpret data requirements and data models that underpin a solution.
Support robust development processes including following best practice for code and release management.
Undertake analysis of systems, business processes, data, and interfaces
Identify production and non-production application issues and fix defects/bugs as required.
Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
Implementing ETL process
Monitoring performance and advising any necessary infrastructure changes
Defining data retention policies
Implementation including loading from disparate data sets, preprocessing using Hive and Pig.
Scope and deliver various AWS and Big Data solutions.
Ability to design solutions independently based on high-level architecture.
Manage the technical communication between the teams.