Third-party testing (revision 35)

Third-party testing is the analysis of a material by a laboratory that is independent of the supplier and of the manufacturer, commissioned by the purchaser and reported to the purchaser. It is distinguished from a supplier's own certificate of analysis not by the analytical methods used, which may be identical, but by who paid for the work and who receives the result.^[1]

The distinction matters because the incentive structure differs. A supplier commissioning analysis of its own material controls which sample is submitted, whether an unfavourable result is published, and how the result is described. A purchaser commissioning analysis of material already received controls none of those things but is testing a sample of unknown representativeness.^[2]

Independent testing is therefore evidence about a sample rather than about a lot, and about a lot rather than about a supplier. Treating a single favourable independent report as a general statement about an organisation is the commonest error made with these documents.^[2]

An independent report establishes, at most, the properties of the material that reached the laboratory. Whether those properties can be extended to the lot the material came from depends on how the sample was drawn, and whether they can be extended to the supplier depends on how many lots have been examined and over what period.^[1]

Three separate questions are involved and are routinely collapsed into one.

1. Is this sample what it claims to be? Answered by identity determination — mass confirmation against the calculated mass, ideally with sequence-level confirmation. A purity figure alone does not answer it.
2. How pure is this sample? Answered by chromatographic purity, which is an area-normalised figure and not a mass.
3. How much active substance is in the vial? Answered only by peptide content together with a water determination and a fill mass. Purity does not answer it.

A report addressing only the second of these — which many do — leaves the first and third open. A specification is judged by which attributes it covers, not by how favourable the covered ones look.^[3]

A sample submitted by a purchaser is a sample of one vial from one order. If a supplier ships from a single lot, the vial may fairly represent it; if a shipment draws from several lots, or if the vial submitted was selected rather than drawn at random, it does not.^[2]

Blind sampling — in which the laboratory does not know the origin of the sample and the supplier does not know a sample has been drawn — removes one class of bias. A chain of custody documenting who held the material between purchase and analysis removes another. Reports that arise from neither are still informative, but they are informative about a narrower thing than they appear to be.

The self-selection problem operates at the level of what gets reported at all. Purchasers who commission testing are not a random sample of purchasers, and results that are published are not a random sample of results obtained. This wiki's own community tally is subject to both effects and says so.^[2]

Four services are referred to throughout the site's body copy, and each has an article: Janoshik Analytical, Medutest, PeptideMeter and VendorInvestigate. They differ in what they do — two are analytical laboratories, one is a results database, one is a documentary review service — and are not interchangeable.

A "PeptideMeter figure" and a "Janoshik result" are therefore not the same kind of object: one is an aggregate over submissions, the other a determination on a sample. Citing them as though they were commensurable is a category error that appears frequently in secondary discussion.^[2]

Where an independent report and a supplier certificate cover the same lot, agreement between them is often presented as validation. It is weaker than that: it establishes that two measurements of the same material agreed, which is expected if both laboratories are competent, and says nothing about whether the material is representative.^[1]

Disagreement is more informative than agreement, but only if the two determinations are comparable. Two area percent figures obtained on different gradients, different columns and different detection wavelengths are not directly comparable, and a difference of a few tenths of a percentage point is within the repeatability of most peptide purity methods. See Analytical method validation.

The practical consequence is that a certificate which does not name its column, gradient and wavelength cannot be compared with anything. Method transparency is what makes a number checkable, and it is a more useful criterion for reading a document than the number itself.^[1] Laboratory competence standards make the same point from the other side: a result is reportable only with the method and conditions that produced it.^[4]

• Certificate of analysis
• Blind sampling
• Chain of custody
• Janoshik Analytical
• PeptideMeter
• Area percent purity
• Peptide content