Blind sampling (revision 22)

Blind sampling is the selection of material for testing under a procedure controlled by neither the party being tested nor the party commissioning the test. In a regulated setting it is what allows a result on a few units to support a statement about a whole lot: the units were drawn according to a written sampling plan, their identity was recorded, and the laboratory did not know whose material it was.^[1][2] The sampling plan is part of the quality system rather than of the analysis, which is why an accredited laboratory can produce an impeccable result on an unrepresentative sample.

Almost no independent testing of research peptides is blind in this sense. The ordinary sequence is that a purchaser buys material, chooses a vial, and sends it to a testing service. Every step of that is under the purchaser's control, and the supplier knows in advance that some proportion of its output will be tested but not which.^[3]

The consequence is narrow and important, and this wiki states it wherever it publishes a test result: a non-blind result describes the vial that was tested. It does not describe the lot, the order, or the supplier, and no quantity of such results aggregates into a statement about any of those, because the selection mechanism that produced them is unknown.

Sampling is the step that connects a measurement to a population. A result is only as good as the answer to the question "of what is this a sample?", and three separate failures can break that connection.

1. Selection by the tested party. If the supplier chooses which vial is tested, the result describes the best vial the supplier could find. No amount of analytical rigour downstream repairs this.
2. Selection by the commissioning party. If the purchaser chooses, the result describes the vial the purchaser chose — which in practice means the one from the order that looked wrong, or the one from the line they care about. Both are informative about that vial and neither is representative.
3. Selection by outcome. If results are submitted to a public tally only when the submitter finds them interesting, the tally is shaped by what motivated submission rather than by what was found. This is selection on the outcome and it is the mechanism that makes community aggregates weak evidence.^[3]

A blind procedure addresses the first two. Nothing addresses the third except collecting every result, which no open submission system does.

Blind sampling requires somebody to buy material anonymously, according to a plan they did not choose, and to pay for testing on it whatever the result. In a regulated supply chain the manufacturer's own quality unit does this because it is obliged to; a regulator does it because it has statutory authority and a budget.

In the research-compound sector there is no obligation and no authority. The parties with a reason to test are purchasers, who are testing their own material because they want to know about their own material, and testing services, which test what is sent to them. Neither is positioned to run a sampling plan.^[3]

A small number of arrangements approximate blinding without achieving it. A purchaser ordering under a name unconnected with any published tally has removed the supplier's ability to identify the order, which addresses failure mode 1 while leaving 2 and 3 untouched. A tally that records every submission received rather than every submission its contributors chose to publicise reduces the third. Neither should be described as blind, and this wiki does not.

The limitation is not a reason to disregard the reports; it is a reason to state what they support. A non-blind, purchaser-selected result on a single vial establishes, without qualification:

• that the material in that vial gave that figure under the stated method;
• that a certificate accompanying it either did or did not agree with an independent determination on the same sample;
• that the vial did or did not carry a lot identifier matching its certificate and invoice.

The second and third are checks on documentation rather than on material, and documentation is the thing a purchaser can actually verify. This is why the supplier articles on this wiki describe documentation practice at length and describe quality only in terms of what the reports say about the samples tested.^[3] It is also why the determinations a purchaser can ask for are enumerated rather than summarised: each is checkable on the document in front of them.^[4]

Three conventions follow from everything above, and they are applied without exception across the twenty organisation articles.

Every published aggregate is labelled at each point of use with its sample size, its collection period and the words that make its provenance visible — self-reported, self-selected. Labelling once at the head of a section is not enough, because paragraphs are split and reordered and the label does not travel with them.^[3]

No aggregate is presented as a ranking. A tally of submissions is a measurement of who submitted, and the difference between two communities' figures for one organisation is evidence about the two communities. PeptidePedia publishes no score and maintains no league table, and the reason is on this page rather than in a style rule.

A comparison is preferred where it is within-subject. Two lines from one organisation, documented differently, compared by the same purchasers, is a better-controlled observation than sixty comparisons across organisations — and it is the only kind of comparison the available evidence supports at all.^[1]

• Chain of custody
• Third-party testing
• Certificate of analysis
• Lot traceability
• Area percent purity

• Project:Sourcing guidelines — the labelling this wiki requires on any community-derived figure.
• Project:Neutral point of view — why a tally may be reported and may not be ranked.