What can you expect in an AI Data Quality Analyst role :

Data Analysis

Quality Audits

• Perform quality audits on annotated datasets to ensure that they meet established guidelines and quality benchmarks.

Statistical Reporting

• Leverage statistical based quality metrics such as F1 score and inter-annotator agreement to evaluate data quality.

Root Cause Analysis

• Analyse annotation errors, trends, project processes, and project documentation to identify and understand the root cause of errors and propose remediation strategies.

Edge-Case Management

• Resolve and analyse edge-case annotations to ensure quality and identify areas for improvement.

Tooling

• Become proficient in using annotation and quality control tools to perform reviews and track quality metrics.

Guidelines

• Become an expert in the project specific guidelines and provide feedback for potential clarifications or improvements.

Continuous Improvement

Automation

• Identify opportunities to use automation to help enhance analytics, provide deeper insights, and improve efficiency.

Documentation

• Develop and maintain up-to-date documentation on quality standards, annotation guidelines, and quality control procedures.

Feedback

• Provide regular feedback that identifies areas for improvement across the annotation pipeline.

Collaboration & Communication

• Cross-Functional Teamwork
• Work closely with key project stakeholders and clients to understand project requirements and improve annotation pipelines.
• Training
• Assist with training annotators, providing guidance, feedback, and support to ensure data quality.
• Reporting
• Provide regular updates that highlight data quality metrics, key findings, and actionable insights for continuous process improvements.

Required Qualifications

• 1+ years of experience as a data analyst with exposure to data quality and/or data annotation - ideally within an AI/ML context.
• Familiarity with the basic concepts of AI/ML pipelines and data.
• Strong analytical and problem-solving skills with an exceptional eye for detail.
• Excellent written and verbal communication skills, with the ability to clearly articulate quality issues and collaborate with diverse teams.
• Ability to work independently and manage time effectively to meet deadlines.
• A strong problem-solver who thinks critically and drives innovation and continuous optimization.
• A quick learner with the ability to work independently in a fast-paced environment.
• A strong focus on detail, balanced against strategic priorities.
• A positive can-do attitude and the ability to easily adapt to new environments.
• Not afraid to speak up.

Preferred Qualifications

• Familiarity with data annotation tools (e.g. Labelbox, Dataloop, LabelStudio etc.).
• Experience working with multi-modal AI/ML datasets (images, videos, text, audio).
• Prior experience in an agile or fast-paced tech environment with exposure to AI/ML pipelines.
• Knowledge of programming languages (e.g. Python).
• Knowledge of the concepts and principles of data quality for AI/ML models and the impacts it can have on model performance.
• Working understanding of common quality metrics and statistical methods used in AI/ML data quality.
• Knowledge of AI/ML concepts and experience with data for AI/ML models.
• Experience in prompt engineering and leveraging LLMs in your day-to-day work.

Education / Certifications

• Bachelor’s degree in a technical field (e.g. Computer Science, Data Science) or equivalent professional experience.

Why you’ll love this role:

• Remote-first flexibility: Work from anywhere you’re most productive!
• Every challenge is new: Work on first‐of‐their‐kind problems at the frontier of AI safety.
• Visible impact: Your architectures directly influence how the world’s leading AI teams mitigate risk.
• Room to grow: Build the solutions function from scratch and scale into leadership as revenue follows.
• Mission‐driven team: Join sellers, researchers, and engineers united by a passion for safer AI.

Work Location / Work Schedule / Travel:

• Open to any PH office (site tagging)
• WFH setup, office on a needs basis only
• Morning or Mid Shift PHT (could change depending on business needs)