The other day, I had the pleasure of being corrected by ever-impressive Milton Packer and I feel obliged to share his teachings with you.
The topic was statins. I said that “only harms detected in randomised placebo controlled trials of statins are the ones to take seriously.” And he said “this is simply not true”. And after I stopped crying, I read the rest of his comments, which were thus:
There are many really serious safety issues that are critically important to patients and which cannot be detected in clinical trials. Trials are designed to evaluate efficacy; they are never large enough to characterize safety, especially for infrequent side effects. Please remember that many life-threatening side effects of commonly used drugs are idiosyncratic. In addition, there are many safety issues that are seen only with longer-term use, and many trials are simply not long enough.
Clinical trials are useful for evaluating safety if (1) the safety issue occurs reasonably early in treatment and with some degree of frequency; and (2) the safety issue is a common human experience (i.e., nausea, headache), making it essential to have a control group. If the safety issue is a rare signature event (with pathognomonic features), a clinical trial is reasonably useless — and unnecessary — in its identification.
There are other complexities here. Take, for example, the case study of azithromycin and sudden cardiac death. Large-scale randomized trials failed to detect any adverse signal at all, and in fact, the point estimate to the effect of azithromycin on sudden death went in a favorable direction. Yet, there may be a tiny group of people who experience QTc prolongation with the drug, and these likely have a well-defined genetic reason. These patients can experience azithromycin-related sudden death as a result of torsade de pointes. But you would never pick this up in a clinical trial, since these trials enroll a broad population and in whom the different forms and etiologies of sudden death cannot be easily distinguished.
He is right, of course, but I have two thoughts. (I know, right! Two thoughts, in one day. Better take a nap pronto.) Also, I am ready to be told I’m wrong again, or at least, somewhat wrong.
First, if a safety issue is so rare that it is missed in a meta-analysis of randomised trials encompassing tens of thousands of patients, is it really that important in the face of the benefits of that medication? His response:
I think you are confusing the characterization of the safety profile of a drug with a calculation of benefit-to-risk relationship. Look at penicillin. Years ago, before the current era of drug resistance, it was life-saving for patients with pneumoccal pneumonia, with estimates that it reduced mortality by > 80%. The drug benefit was so large that you could have demonstrated a striking benefit in a trial of fewer than 100 patients. But the drug causes life-threatening anaphylaxis in 1 of 2000 patients. You would never have detected a reaction that infrequent in an efficacy trial of 100 patients. But it would make no sense to suggest that anaphylaxis was “really not that important”. Anaphylaxis is critically important. However, it is so rare that — in the absence of a known history of penicillin allergy — you would have always chosen to give the drug to someone with pneumococcal pneumonia. The benefit-to-risk relationship was strongly in favor of treatment, but even though the adverse effect of anaphylaxis is really quite important.
And second, how do we as researchers convey to clinicians, patients, and the general public, that when people who take medication experience undesirable things, they should be careful when attributing those experiences to that medication. It’s so easy to attribute side effects to medication when the leaflet in the box is as long as your arm and forget about the list of things that are caused by old age, obesity, exercise, alcohol, coffee, arthritis etc. How do we counter the overwhelming influence that anecdotes have on us humans? It is instinct to believe trusted friends and relatives experiences over your doctor who has known you for all but 5 minutes. Is it possible to get the concept of a n-of-1 trial like SAMSON into primary care as a way of testing whether a side effect is attributable to a medication? His response:
This is an entirely different topic, which is not an issue that is related to the design or interpretation of clinical trials.