Jump to a company, a ranking or a tool
Updated October 2026
Mercor is the largest vendor by reported run rate, above $2B[2], but confirmed a cyberattack in March 2026[4]. Surge AI reported $1B to $1.2B in 2024 revenue with no outside investors[14], and Scale AI is 49% owned by Meta[10]. Few vendors publish quality or security evidence, so most scores rest on two or three criteria, and we show exactly which.
Pays people from its career network as AI training fellows.
Best forLabs that want to draw on a large career network
Watch out for: Little published on vetting or security1
Data annotation and collection at scale.
Best forBuyers who want a listed company with public financials
Recruits experts for AI training work and licenses company operating data.
Best forLabs that also want licensed company data alongside expert work
Watch out for: Net revenue reported far below the gross run rate7
Human participant platform for research and AI evaluation.
Best forTeams that need human participants for research and evaluation
Watch out for: Revenue figure is an outside estimate9
Not rated yet
How we rateManaged data labeling and human data for AI teams.
Best forTeams with no competitive overlap with Meta
Watch out for: Ownership conflict: Meta holds 49%, and competes with many of Scale's customers.10
Not rated yet
How we rateTraining data provider for AI teams.
Best forEnterprise teams that want a funded, established data partner
Watch out for: Too little published to score most criteria12
Human data for model training, sold to AI labs.
Best forLabs that want a large vendor with no outside owners
Watch out for: Little published on quality or security14
Not rated yet
How we rateSupplies coding data to OpenAI and other LLM builders.
Best forLabs buying coding data
Not rated yet
How we rateAI training data for model builders.
Best forTeams comparing alternatives to Scale AI
Not rated yet
How we rateHuman data and evaluation for AI teams.
Best forTeams comparing independent data vendors
Not rated yet
How we rateExpert depth and vetting25%
Quality systems20%
Security and neutrality20%
Scale and speed20%
Transparency15%
A score is published only when criteria worth at least 35% of the weight are evidenced. The overall score is the weighted mean of the evidenced criteria. Commissions are never an input. Full methodology
FAQ
A human data vendor supplies the people whose judgments, answers and corrections are used to train and test AI models. Some, like Mercor, sell expert work to AI labs at very large scale. Others, like micro1, also license operating data from companies. This site ranks both kinds for the reader each one serves.2,19
Mercor is the largest by reported figures, with a gross run rate above $2B in June 2026. Surge AI's 2024 revenue is reported at $1B to $1.2B, and Handshake's gross ARR at about $1.15B. micro1 reported a $500M gross run rate in August 2026, Snorkel $375M, and Prolific about $350M by an outside estimate.2,14,20,1,7,12,9
Most of these are gross figures from unnamed sources, and gross and net revenue can differ widely. TechCrunch put micro1's net revenue at $150M-$200M against its $500M gross run rate.
Each audience has its own published rubric of five weighted criteria, and every sub-score cites the facts behind it. A score appears only when we can evidence at least 35% of the rubric's weight; otherwise the card says there is not enough public data. Commissions never enter the calculation.
Ownership is the first thing to check. Meta has held 49% of Scale AI since June 2025, which matters to any lab that competes with Meta. Surge AI says it has never taken outside funding, so no lab owns part of it. Ask every vendor for its owners, its SOC 2 report and its incident history before a pilot.10,15
Mercor confirmed a cyberattack on 31 March 2026, tied to the compromise of the open-source LiteLLM project, and several class actions were filed in the Northern District of California on 1 April. Mercor later called the affected data a very limited subset. No reliable count of affected people has been published.4,5
Mercor states a SOC 2 Type II attestation on its data page. None of the other companies in our current sources publishes one where we could check it. That does not mean they lack one: ask each vendor for its report under NDA, and treat a refusal as a signal.21