QOVES — Independent NLP Audit for Ingredient Classification

Client and Product Context

QOVES is an Australian company focused on personal aesthetics, beauty, and skincare analysis. The company was developing proprietary technology to analyse and classify skincare ingredients using NLP. They had engaged a third-party development team to build an AI/NLP prototype intended for production use. After six months, the delivered system failed to meet expected technical and functional standards.

The Challenge

Scalexa was engaged to assess whether the existing NLP solution was technically sound, fit for purpose, and professionally executed. The audit needed to produce findings that were technically rigorous, evidence-backed, and suitable for use in legal and contractual discussions—all within a constrained timeframe and without rewriting the system.

• Determine whether the data sources were sufficient in size and quality
• Assess whether the models and pipelines were appropriate for the stated objectives
• Evaluate whether the produced outputs could reasonably be expected from the inputs and timeframe
• Produce findings suitable for legal and contractual proceedings

What Scalexa Did

Scalexa conducted a methodical, evidence-based audit of the entire NLP pipeline. The approach focused on traceability and defensibility of findings.

• Reviewed the entire NLP pipeline: source data (raw, processed, filtered), feature extraction, preprocessing, model selection, and training approach
• Matched code paths to observed outputs to assess correctness and traceability
• Evaluated whether the data series was sufficiently large and representative to support the intended classification task
• Assessed alignment between contracted scope, claimed functionality, and actual technical implementation
• Identified shortfalls, inconsistencies, and architectural weaknesses
• Documented findings with verifiable technical evidence
• Delivered results in a formal written report suitable for legal review
• Participated in stakeholder feedback and clarification discussions

Scope of Audit

The audit covered the full technical scope of the NLP system without embellishment or speculation.

• Domain: Natural Language Processing (NLP)
• Use case: Ingredient classification for skincare and beauty products
• Artefacts reviewed: Codebase, datasets, models, outputs, tooling
• Focus areas: Data sufficiency and validity, model appropriateness, output correctness, engineering discipline and reproducibility

Technical Implementation: How the AI Technical Audit Was Run

The AI technical audit followed a five-stage process, each stage producing written evidence that could be traced back to a specific file, dataset, or model artefact. QOVES does use AI: the product relied on an NLP classification pipeline that mapped free-text cosmetic ingredient names to a controlled taxonomy of ingredient functions and safety categories. Our task was to establish whether that pipeline was built to a professional standard.

• Stage 1 — Data forensics: counted labelled examples per class, measured class imbalance, checked for duplicate and leaked rows between train and test splits, and assessed whether the ingredient corpus was large and representative enough to support supervised NLP classification at the accuracy claimed.
• Stage 2 — Preprocessing and feature review: traced tokenisation, normalisation of chemical and INCI ingredient names, stop-word handling, and vectorisation choices, checking each against the stated classification objective.
• Stage 3 — Model appropriateness: reviewed model selection, hyperparameters, and training scripts to judge whether the chosen architecture was a reasonable fit for a multi-class text classification task of this size, and whether simpler, stronger baselines had been evaluated.
• Stage 4 — Output traceability: re-ran code paths against supplied data and matched produced outputs to the code that generated them, isolating results that could not be reproduced from the inputs in the contracted timeframe.
• Stage 5 — Engineering discipline: assessed version control history, environment reproducibility, evaluation methodology, and the absence or presence of a held-out test protocol.
• Evidence handling: every conclusion was tied to a file path, dataset statistic, or reproduced run, with opinion clearly separated from demonstrable fact so the report could withstand legal scrutiny.

Legal and Professional Considerations

Scalexa acted as an independent third party throughout this engagement. Conclusions were evidence-based and documented. The engagement focused on assessment, not remediation. Care was taken to avoid speculative or unverifiable claims. All findings were structured to withstand professional scrutiny.

Technologies

Why It Matters

If you require an independent technical audit of an AI or NLP system—whether for validation, risk management, or legal clarity—Scalexa can provide an objective, evidence-based assessment. Our senior team has experience operating at the intersection of engineering, data science, and legal scrutiny, delivering defensible conclusions that stand up to professional review.

Services Behind This Project

The same senior teams that delivered this work:

• AI & Machine LearningProduction ML, LLM and computer-vision systems
• CybersecuritySecurity architecture, audits and hardening