Job Specific Duties & Responsibilities:
* Develop & optimize ETL/ELT pipelines using Databricks, PySpark, and SQL
* Migrate on-premise data to AWS cloud, ensuring integrity and performance
* Design and implement scalable data models to support analytics and business intelligence
* Manage and develop stored procedures for efficient data processing
* Perform data integration & transformation across various cloud and on-prem data sources
* Enhance data analytics capabilities by processing and structuring data for reporting
* Optimize AWS-based data architecture (e.g., S3, Redshift, Glue, Lambda, Athena)
* Ensure data quality, governance, and security across the pipeline
* Collaborate with data scientists, analysts, and business teams to drive insights
Required Skills, Experience & Qualifications:
* 5+ years of experience in Data Engineering with a focus on cloud platforms
* Strong expertise in Databricks (PySpark, SQL, Delta Lake)
* Proficiency in AWS services (S3, Redshift, Glue, Lambda, Athena, Step Functions)
* Advanced skills in Python & SQL, including writing stored procedures
* Experience in large-scale data migration to the cloud
* Knowledge of data modeling, warehousing, and analytics
* Strong experience in data integration, transformation, and ETL pipelines
* Understanding of data governance, security, and performance optimization