More than 80 years ago, the 1936 anti-drug propaganda film flick “Reefer Madness” chronicled the supposed “dangers” of marijuana – depicting characters who descended into a spiral of murder, attempted rape, and insanity because of cannabis. The film was a reflection of social anxieties and the “pot panic” of that era, which was, by and large, a byproduct of misinformation, political propaganda, and poor science.
Fast forward: The studies that exaggerated, or entirely fabricated, negative health effects from cannabis that arose during this era have largely been debunked, or the studies have been called out for being ill-constructed or flawed.
But even in the world of cannabis today, we still see so much bad science. Why do we care, and why does it matter?
So much of this boils down to the single biggest problem facing our industry today: potency inflation.
Potency inflation occurs because high THC percentage cannabis products sell for a higher price. This incentivizes bad actor cannabis businesses (brands, cultivators, distributors) to shop around (called ‘lab shopping’) for similarly bad actor testing labs that are willing to use bad science and/or fraud to doctor cannabis sample results to show the highest THC percentage possible. This data is then presented on a Certificate of Analysis (or CoA) that links to cannabis products on shelves.
Lab shopping has led the worst actor cannabis brands and labs to perform, while high efficacy brands and science-first labs have suffered. This creates an unnecessary, unsafe, and vicious cycle of misinformation where the end customer is put at risk. And, as many of our colleagues have thoughtfully pointed out, potency inflation has become so prevalent because there have been no negative repercussions for cheating.
We believe that consumers should know what’s in their cannabis. Consumers know the exact level of every vitamin in their food, the ABV of every alcoholic drink, and the full caloric breakdown of a can of soda. Cannabis shouldn’t be any different – but that’s not the reality today, even in our regulated market.
We believe that consumers should know what’s in their cannabis.
To overcome this, we support new state-based regulations that prioritize well-designed comparative studies and enforce consequences when cheating happens. Requiring comparative studies can effectively establish which labs in our industry follow the rules, and which don’t. But the results of a poorly-designed study can also be dangerously misleading.
What makes a proper study design?
And what’s the right scientific approach? When reviewing data from a comparative study, you must keep three guiding principles in mind:
- Who is conducting the study? It is not uncommon for studies to be funded by entities with a particular agenda. The most meaningful studies must be conducted by unbiased third parties with no conflicts of interest and nothing to gain from any particular result.
- How was the study conducted? The underlying methods are the most important yet highly varied element when reviewing any study. The best studies are always conducted impartially by someone with knowledge of the corresponding field.
- Is it statistically significant? For any study to have merit, it must include a large enough dataset to where the results of the subset studied would reflect the results of the larger group.
In our line of work, all ISO 17025-accredited laboratories, a critical and should-be standard level of cannabis lab accreditation, must participate in a Proficiency Test (PT) at least annually. A PT consists of testing a sample containing analytes at concentrations unknown to the laboratory but known to the PT provider. The results from these PT studies are then compared between laboratories to determine acceptance criteria.
When conducting interlaboratory comparisons, much consideration must be given to the starting material. If an improperly prepared PT sample were distributed, it could have drastic effects on the ISO accreditation of participating laboratories and pose concerns for public health. Therefore, these PT providers must achieve their own ISO 17043 accreditation in order to produce PT samples.
Once the above is taken care of, here are the core questions to ask to effectively evaluate an interlaboratory study in the cannabis space:
- How homogenous was the sample studied? Any study where the initial sample is not homogenized prior to distribution to participants should be taken with a grain of salt as there are no assurances that each participant received the sample to test. To ensure a level playing field, the sample should be thoroughly homogenized to where differences between subsets would be negligible.
- How was the sample characterized? If the expected results of a study were determined using one or two data points, it is baseless, by definition. Most PT providers do not define a “true” value but instead opt for an “assigned value” surrounded by a range of acceptable values. This is because unless you are spiking a known concentration of analytes onto a blank matrix, the actual concentration is unknown. The assigned value is typically determined through numerous repeat analyses of the same sample often across multiple instruments, analysts, and facilities. The variance between these replicate analyses often determines the acceptable range.
- How was the data analyzed/presented? It is always important to ensure the data is represented in its entirety. Too often, researchers “cherry-pick” data and choose to display results that support their assumptions while hiding results that might yield conclusions to the contrary.
Conclusion
Cannabis labs need to be held accountable for the results that they deliver, and effectively undertaking a proper comparative study can help achieve that goal, which is something we at SC Labs strive for after over a decade of working to develop industry-leading standards. We do this to protect our customers – inclusive of cannabis businesses and importantly consumers – as well as to ensure the long-term viability of this industry we love. The above guidelines set a protocol for ensuring good science wins, and forms a roadmap for uncovering the bad actors in our industry.
About the Author
Josh Wurzer, SC Labs’ Co-founder and Chief Operating Officer, is the recent past President of the Pacific Southwest division of the AOAC, and is currently participating in the U.S. Pharmacopeia (USP) Cannabis Expert Panel; the AOAC International Cannabis Working Group and is a founding member of the CASP committee; the American Council of Independent Laboratories (ACIL) cannabis committee; the National Institute of Standards and Technology (NIST) Cannabis working groups; the American Chemical Society (ACS) Cannabis Chemistry Subdivision as well as serves on several other cannabis, hemp, and chemical trade association committees.