Sr Data Engineer (Azure)
Position: Permanent
Work Location: Telford, UK
Frequency: 2 days work from office, Telford
Day Rate: £50K per annum
Experience Required: 10+ years in Azure Data.
Responsibilities:
* Design, develop, and maintain scalable ETL pipelines using Azure Services to process, transform, and load large datasets into AWS Datalake or other data stores.
* Collaborate with cross-functional teams, including data architects, analysts, and business stakeholders, to gather data requirements and deliver efficient data solutions.
* Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.
* Work together with data scientists and analysts to understand the needs for data and create effective data workflows.
* Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
* Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations.
* Implement data validation and cleansing procedures to ensure the quality, integrity, and dependability of the data.
* Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
* Monitor and resolve data pipeline problems to guarantee consistency and availability of the data.
Key Skills Required:
* Experience in designing and hands-on development in cloud-based analytics solutions.
* Expert level understanding of Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service.
* Designing and building of data pipelines using API ingestion and Streaming ingestion methods. Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is desirable.
* Strong experience in common data warehouse modelling principles including Kimball and Inmon.
* Knowledge in Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics, and Power BI is desirable.
* Working knowledge of Python is desirable. Experience developing security models.
#J-18808-Ljbffr