The Role
We are seeking a Data Quality Assurance Engineer to join our team and support a critical data migration and consolidation initiative. As part of this effort, the Data Quality Assurance Engineer will be integral to ensuring data accuracy, consistency, and reliability throughout our systems. Working closely with the Senior Data Engineer, this role will focus on validating ETL processes, identifying data quality issues, and establishing robust data QA practices that enhance our Snowflake-based data warehouse as it continues to evolve.
The ideal candidate has hands-on experience with data validation across the full data lifecycle, particularly with data transformations, cleansing, and testing in a fast-paced, high-stakes environment. Familiarity with tools like Snowflake and FiveTran, combined with strong SQL skills and a sharp eye for detail, will enable the Data Quality Assurance Engineer to support data engineers effectively and uphold high data quality standards.
This role requires a proactive approach to data quality, with the ability to work independently and collaboratively to diagnose, document, and resolve data discrepancies. Experience in the financial lending sector is a plus, as this knowledge will enable you to address industry-specific data challenges more effectively.
You will be involved in:
Data Quality Assurance and Automation
* Perform rigorous data validation and testing to support data migration efforts, ensuring high standards of data quality and integrity.
* Collaborate with data engineering on ELT testing, specifically in identifying and troubleshooting errors in data movement and transformation processes.
* Establish and implement automated data validation frameworks that ensure data accuracy, consistency and reliability across our data pipeline.
* Create and maintain CI/CD-integrated QA workflows to support real-time testing and automated quality checks, focusing on Azure and Snowflake infrastructure.
* Develop automated test cases for data validation, focusing on critical workflows, data transformations, and API integrations within both Snowflake and Azure.
Collaboration & Support for Data Engineering
* Work closely with the Senior Data Engineer to set and maintain quality standards for the Snowflake data warehouse, supporting scalability and reporting needs.
* Participate in data architecture discussions, providing QA insights on best practices for data collection, transformation, and loading processes to maintain end-to-end quality.
* Ensure data reliability and accuracy by proactively identifying potential data discrepancies and issues, particularly API data integration, CI/CD workflows.
* Develop reusable, modularised test scripts in Python and SQL to streamline testing across various stages of the data pipeline.
* Conduct regular data cleansing activities, identifying, documenting, and addressing data anomalies within the financial lending dataset.
* Partner with business stakeholders to understand data needs and ensure that data integrity meets end-user requirements for accurate reporting and analysis.
* Support the development of data governance and QA processes to streamline issue resolution and establish best practices for ongoing quality control.
Documentation & Process Improvement
* Document QA processes, test cases, and key findings, and quality control standards to establish a knowledge base for quality assurance.
* Proactively contribute to the development of data governance standards and practices, including version control and GitHub workflows to support efficient change management.
Preferred Knowledge & Experience
Data Lifecycle QA
* Solid understanding of the data lifecycle, including experience with data collection, transformation, cleansing, validation, and testing in complex environments.
* Familiarity with data integration tools like FiveTran and proficiency in SQL for testing data accuracy and consistency.
Technical Proficiency
* Hands-on experience with data validation in Snowflake or similar cloud data warehouses, including data testing and quality checks in SQL views.
* Proficiency in SQL and Python for advanced data validation and data quality assurance automation.
* Knowledge of Azure and Azure DevOps for managing QA within CI/CD pipelines, including experience with containerised environments.
* Familiarity with automated data validation and monitoring tools within Snowflake and Azure, focusing on real-time error detection and alerting.
* Knowledge of data automation tools and scripting for streamlined data QA processes.
Financial Sector Familiarity (Preferred)
* Experience in financial lending or similar sectors, understanding industry-specific data requirements and quality assurance practices.
Collaboration & Communication
* Strong interpersonal skills for working effectively with data engineers, business users, and stakeholders to support data quality objectives.
* Strong communication skills to document findings, explain complex QA processes and contribute to architecture discussions.
Qualities and Competencies
* Meticulous attention to detail, ensuring high levels of data accuracy and reliability.
* Problem-solving mindset with a collaborative approach to support team objectives.
* Ability to work both independently and as part of a cross-functional team.
* Excellent communication skills for documenting findings and explaining complex QA processes.
* Strong organisational skills with an emphasis on project management and prioritisation.
* Proactive, results-driven attitude with a commitment to maintaining data integrity.
* Inquisitive and curious personality with a passion for data quality.
#J-18808-Ljbffr