The rate of suicide has been rising for 16 years and it is the 10th leading cause of death in the United States. Most suicides occur in the context of a psychiatric disorder (Bachmann, 2018), and yet the effect of many medications, including psychotropic medications, on suicide risk is intensely debated. The objective of this study is to develop a statistical surveillance methodology based on generalized mixed-effects regression models applied to analysis of large-scale medical claims and medical records that identifies drugs associated with increased and decreased risk of suicidal events. To this end, we use a within-person incident-user cohort design to simultaneously examine the relationship between 922 drugs and 43,978 suicidal events (suicide attempts and intentional self-harm, including fatalities if they resulted in a medical claim) using medical claims for private health insurance (MarketScan) from 2003–2014, for over 150 million people. Suicidal events were identified based on the following ICD-9 codes (E950-E959). Analysis of medical claims data for 922 drugs revealed statistically significant associations with suicidal events, including 10 drugs with increased rates and 44 with reduced rates following exposure. Among the strongest increased risk signals identified were alprazolam, butalbital, hydrocodone, and the codeine/promethazine mixture, and among the potentially most protective drugs were folic acid, mirtazapine, hydroxyzine, disulfiram, and naltrexone. Thirty of the 44 drugs with decreased risk are approved psychotropic medications providing both a degree of validation of the method and reassurance to clinicians about the effectiveness and safety of these drugs in suicidal patients. High-dimensional drug safety surveillance using extensive observational data is feasible and generates statistically and clinically significant signals of possible risks and benefits of drugs on suicide risk.
Keywords: empirical Bayes, drug safety, suicide
Suicide is the 10th leading cause of death in the United States (CDC, 2019), and the rate has risen for the past 16 years. Medication-induced suicide has been a topic of theorizing and U. S. Food and Drug Administration (FDA) scrutiny for decades. The FDA requires a black box label for more than 130 medications regarding the risk of suicidal ideation or behavior (Au, 2016; Lavigne et al., 2012), but leaves prescribers, patients, and families to negotiate the risks of prescribing with little information about its relative magnitude. Based on the most recent data (2009), more than 540 million 30-day-supply prescriptions of medications labeled for suicide risk were filled in the United States, and nearly 5 million in the U.S. Department of Veterans Health Affairs (VHA) out of 6 million veterans (Lavigne et al., 2012). Few (if any) studies, have established mechanisms for suicide risk in a drug class or specific drug (Mann et al., 1993). Most pharmacoepidemiologic studies have limited comparisons to a single drug class (e.g., antidepressants, antiepileptics) (Gibbons, Hur, Brown, & Mann, 2009) or even a single drug (varenicline) (Cunningham et al., 2016; Gibbons & Mann, 2013; Gunnell, Irvine, Wise, Davies, & Martin, 2009), limiting our knowledge of relative suicide risk. Lack of knowledge with which to address fears about medication usage and suicide risk has led to underprescribing (for example, varenicline, antidepressants) affecting overall population health (Lu et al., 2014; Tippett & Chen, 2015). The need for improved methods to detect the risks and benefits of medications with regard to suicide risk is a significant policy, public health, and clinical priority (National Action Alliance for Suicide Prevention, 2014).
Traditional postmarketing surveillance of pharmaceuticals has been based on pharmacovigilance studies using spontaneous reports of adverse events (AEs) submitted to pharmaceutical companies and regulatory agencies (e.g., FDA MedWatch). Despite creative statistical approaches to the analysis of such data (DuMouchel, 1999; Gibbons & Amatya, 2015), several critical limitations remain. First, the population at risk is unknown so the rate of adverse drug reactions (ADRs) is inestimable. Second, we know little about diagnoses, indications for treatment, prior treatment, and concomitant medications. Third, lack of basic demographic information (age and gender) limit generalizability. Fourth, there is confounding by indication; suicide is most prevalent among people with major depression, depression is treated with antidepressants, hence there is a noncausal association between antidepressants and suicide. Additional problems include systematic underreporting, questionable representativeness of patients, media stimulation of reporting, duplication of reports, and extensive missing information (Health Affairs, 2015). While there have been some useful results for rare AEs (Burke et al., 2006; Wysowski & Schwartz, 2005), they are of limited value for more prevalent AEs or rare AEs related to the indication for treatment. They also provide no information on drugs that may decrease risk.
In this study, we apply previously developed generalized mixed-effects regression models (see Hedeker & Gibbons, 2006) to the problem of identifying signals related to the safety of pharmaceuticals. We refer to the approach as High Dimensional Empirical Bayes Screening (iDEAS). iDEAS overcomes many of the problems of traditional pharmacovigilance methods for drug safety screening. Instead of relying on spontaneous reports we use commercial claims data (MarketScan: Hansen, 2016) to estimate both risks and benefits of medications on suicidal events. We apply iDEAS to simultaneous screening of risks and benefits of 922 prescription drugs (all prescription drugs with 3,000 or more filled prescriptions in 2014 in the MarketScan database) in terms of suicidal events defined as suicide attempts and intentional self-harm, which include fatalities if death occurred in a hospital or emergency department and resulted in a medical claim. These data are person-level and longitudinal so that one can evaluate the timing of the drug exposure in its relationship to an AE of interest. These databases cover over 150 million lives, so in combination they are more strongly representative of the U.S. population. These are observational data and the purpose of these analyses is identifying safety signals, which must be independently confirmed in either RCTs or large-scale pharmacoepidemiologic studies using statistical and design tools that can provide causal inferences. Advantages of this approach are (a) it is person-level, (b) we know the population at risk (those who took the medication), (c) the comparisons are within-person (i.e., before and after exposure), eliminating many of the biases related to between-subject selection effects (Gibbons & Amatya, 2015), (d) it can be used to simultaneously screen all drugs for one or more AEs, and (e) it is based on large databases so that even rare ADRs can be identified.
In this study we used data from the MarketScan Commercial Claims and Encounters database (Hansen, 2016). The MarketScan data (provided by IBM Watson) include inpatient, outpatient, and prescription claims from more than 100 insurers in the United States (146 million unique enrollee observations since 2005). We studied all 922 prescription drugs in an incident user cohort (2003–2014) that contained more than 3,000 prescriptions in 2014 in the MarketScan database and examined the rates of suicidal events three months prior and three months after the prescription fill. Suicidal event ICD-9 codes were E950-E959, which include fatal and nonfatal suicide attempts and intentional self-harm, that resulted in a medical claim. Our analysis was based on an exposure-only design so that, for each drug, we considered the data from the subset of patients who took the drug in question. Records in which both the prescription and the AE occurred on the same day were eliminated to reduce the possibility that the suicidal event led to treatment with drugs used to treat neuropsychiatric illness. Patients with a prior suicidal event in the year preceding the index prescription 180-day interval were excluded, thereby limiting our analysis to newly emergent suicidal events.
We used a mixed-effects logistic regression model (Hedeker & Gibbons, 2006), from which we estimated empirical Bayes (EB) odds ratios (ORs) and 95% confidence intervals (CIs), adjusted for multiple comparisons for the individual drug effects. In our model, the drug effects are estimated by the EB estimate of the main effect of time for each drug, the exponential of which is the OR for the increase (or decrease) in likelihood of the AE following the initial prescription fill for the medication. Other covariates such as age and sex can be treated as either fixed and/or random effects and their interactions with time allow these covariates to modify the drug-specific effects. The interpretation of the EB OR is the increased or decreased likelihood that the adverse event will occur following drug exposure. ORs less than 1.0 are associated with lower than expected postexposure events and ORs greater than 1.0 are associated with higher than expected postexposure events. Medications for which 1.0 is not contained within the experiment-wise 95% CI (adjusted for multiplicity produced by the simultaneous analysis of 922 drugs) are considered statistically significant. In the following section we provide a more detailed presentation of the statistical model.
Let denote a drug and denote a subject which took that drug. Assume that there are drugs, denoted and claimants for drug , denoted. Let if claimant j who took drug experienced the AE, else . For the patients taking drug we can further stratify the data by time (before versus after the prescription fill), and other relevant covariates (e.g., age and sex) and denote these covariates as , a covariate vector (including for the intercept and any relevant interactions can be added as well). Since our interest is in case-mix adjustment for the covariates (e.g., time, age, and sex) at the drug level, the elements of are treated as both fixed and random effects in the following mixed-effects logistic regression model
This model results from the assumption that the vector of random effects follows a multivariate normal distribution with mean vector and variance-covariance matrix. To aid in estimation we standardize the random effects, where is the Cholesky factorization of . Estimation of rather than is more stable as variance terms approach zero.
We estimate the parameters of the model using a combination of maximum marginal likelihood (MML) for the fixed-effects and variance-covariance matrix of the random effects and EB estimation for the drug-specific effects. Adjustment for overall (over all drugs) covariate effects can be obtained by adding the covariates as fixed-effects to the model (e.g., do women have more suicidal events in general?). Drug-specific adjustment requires both fixed and random covariate effects (e.g., do women taking fluoxetine have more suicidal events than men?). The random time by sex interaction can be used, for example, to determine if the difference in suicidal event risk of the AE before and after treatment with fluoxetine differs between men and women. We can write the model in terms of the logistic cumulative distribution function (cdf) as
where . Note that we use the same for both fixed and random parts of the model for this application, but as noted above this is not a requirement of the model.
The conditional likelihood is then
Integrating over the random effect distribution gives the marginal probability for in the population as
Specific details of the estimation procedure are given in Hedeker and Gibbons (2006). With , the likelihood equations can be evaluated numerically using Gauss-Hermite quadrature (Stroud & Sechrest, 1966). When higher dimensional case-mix adjustment or stratification is required, maximum a posteriori (MAP) estimation can be used to evaluate the marginal likelihood in terms of the mode of the posterior distribution. Unlike the frequentist method of maximum likelihood, MAP incorporates a prior distribution over the quantity of interest in the objective function. MAP estimation for nonlinear mixed-effects models was first proposed by McGilchrist (1994). Viable alternatives are the Laplace approximation (i.e., Taylor series expansion of the integrand to bypass the integration) or a full Bayesian approach using Monte Carlo simulation estimation. Finally, the method can be further generalized to a more complete longitudinal analysis using a discrete time survival model (Gibbons & Amatya, 2015), in place of the logistic regression model, albeit at considerable computational expense because person-level data are required in place of the number of events and population at risk.
For this application, we seek case-mix adjusted drug-specific estimates of the time effects (i.e., difference before and after exposure). Here we define the expected a posteriori (EAP) or EB estimate (Hedeker & Gibbons, 2006) as
the posterior mean, which can also be evaluated numerically using quadrature. The variance of the posterior mean is
which can be used to derive confidence bounds for the estimated drug-specific effects. The elements of are drug-specific residuals for the overall case-mix adjusted treatment (exposure) effect and can be used to estimate strata-specific treatment effects (for example stratified by age and sex).
The exponential of various linear combinations of the EB estimates ( ) are used to estimate potential ADR-specific odds ratios which we use to identify both significant protective and harmful drug-AE associations, both overall and within specific age and sex subpopulations. To account for multiplicity, we adjust the confidence intervals for, the total number of drugs examined using a simple Bonferroni type correction (i.e., 0.05/), which in this case N = 922. This yields an experiment-wise confidence level of 95%.
As an illustration, consider the data for alprazolam from the MarketScan database for which there were a total of 4,719,202 patients that made took alprazolam, 1,861 who had a suicidal event after filling a prescription and 1,035 who had a suicidal event prior to filling a prescription for alprazolam. The simple relative risk is (1861/4719202)/(1035/4719202)=1.80. Table 1 reveals an estimated OR = 1.72 (1.47, 2.01).
# of Suicide Attempts
OR (95% CIs)
# of Subjects
Codeine Phosphate/Promethazine Hydrochloride
Let be the event that a person experiences an AE of interest and be the time of its occurrence. Note that for recurring AEs, may be a vector. Let be the event that a drug is filled by the patient and the time that the patient filled the prescription. Denote if and if . For drug we have
where is the odds of occurring prior to and is the odds of occurring on or after . Our goal is to estimate the OR,
which we take as evidence for a signal of a potentially harmful or beneficial association if its multiplicity adjusted confidence interval does not contain the value 1.0. The statistical validity of this estimator is based on two key assumptions. First, we make the strong assumption that and are identically and independently distributed conditioning on and . While this assumption may be reasonable for drugs that are not used for the treatment of disorders that are related to the AE of interest, this may not be true for drugs that are used for the treatment of indications that convey increased risk for the AE of interest (e.g., depression and suicide). In our example for suicidal events, drugs used to treat neuropsychiatric indications may be prescribed in response to a suicidal event to treat the underlying neuropsychiatric disorder. For this reason, it is not uncommon for for drugs that are used in the treatment of conditions that convey risk for the AE. To help reduce the association between and we exclude cases in which . Sensitivity analyses can be conducted to extend the exclusion of AEs that occur within a week of initiating the drug.
We note that the bias produced is unidirectional, in that it can produce artificial protective associations, but cannot produce artificially harmful associations, where . It is also not a factor for drugs that are not used for the treatment of disorders that are related to the AE directly, for example, diabetes drugs and suicidal events.
An additional assumption that is made in all pharmacoepidemiologic studies is that the patient who filled the prescription actually took the medicine and did so compliantly.
A question arises as to possible bias and shrinkage of the EB estimates, such that the intended coverage is not what is achieved in practice. To this end we conducted a limited simulation study to examine the actual coverage level of the methodology based on simulated data. To this end, we simulated data for 1,000 drugs and one million patients per drug. Under the null hypothesis of OR = 1.0, we assumed a base-rate probability of a suicidal event of 0.000113 (=exp(-9.093) from the overall model) in each time period (pre and post), and a random effect variance of 0.05 for the time effect (pre versus post difference). The simulation was then replicated 1,000 times. The estimated coverage rate (adjusted for multiplicity) was 96% with a root mean square error (RMSE) for the OR of 0.153. We next examined the alternative hypothesis in which 95% of the data were simulated under the null and 5% of the data were simulated under OR = 2.0. For the 95% of the data simulated under the null, the simulated Type I error rate was 4.2% with an average RMSE of 0.197. For the 5% with simulated with OR = 2.0, power was 42.8%.
Table 1 provides a summary of all medications that were associated with increased postexposure risk of suicidal events, Table 2 lists medications that were associated with decreased postexposure suicidal event risk, and Table 3 lists medications associated with increased risk excluding overdose. A list of results for all 922 medications is provided in the Supplement.1
# of Suicide Attempts
OR (95% CIs)
# of Subjects
# of Suicide Attempts
OR (95% CIs)
# of Subjects
Ten drugs were associated with statistically significant higher risks of suicidal behavior (Table 1). Among these were two anxiolytic benzodiazepines, alprazolam and diazepam; the opioid and analgesic/opioid narcotic mixtures acetaminophen/hydrocodone bitartrate and codeine phosphate/promethazine hydrochloride; the barbiturate-stimulant mixture acetaminophen/butalbital/caffeine; the muscle relaxants cyclobenzaprine and carisoprodal; the steroid prednisone; the phenothiazine promethazine hydrochloride (used to treat or prevent nausea and vomiting caused by anesthesia or surgery); and the antibiotic azithromycin.
Table 2 displays the 44 medications associated with statistically significant decreased risk of suicidal events following exposure. The list includes a large group of FDA approved antidepressants: selective serotonin reuptake inhibitor (SSRI) antidepressants citalopram, escitalopram, fluoxetine, and sertraline; the mixed serotonin and norepinephrine reuptake inhibitors (SNRI) venlafaxine, desvenlafaxine, and duloxetine; bupropion, an aminoketone that inhibits norepinephrine reuptake; mirtazapine, a noradrenergic alpha2-adrenergic antagonist; trazodone, a serotonin 5-HT2A receptor antagonist and reuptake inhibitor; and the tricyclic doxepin. The list also contains several antipsychotic medications, including both the typical antipsychotics haloperidol and perphenazine, and the atypical antipsychotics aripiprazole (also produces substantial 5-HT2A antagonism), asenapine, clozapine, quetiapine, risperidone, lurasidone, paliperidone, ziprasidone, and olanzapine. Additional medications include alcohol/drug abuse treatment drugs acamprosate, buprenorphine/naloxone, naltrexone, and disulfiram; antimanic or mood stabilizer drugs (including antiepileptic drugs [AEDs] used in the treatment of bipolar illness and chronic pain), including lithium carbonate, carbamazepine, gabapentin, lamotrigine, oxcarbazepine, and divalproex sodium; the ADHD medication guanfacine hydrochloride; the anxiolytic buspirone; the antihistamine hydroxyzine; the vitamin folic acid; the Parkinson’s disease anticholinergic medication benztropine mesylate, which is also used to ameliorate extrapyramidal symptoms in patients taking antipsychotics; the anti-hypertensive medications amlodipine, clonidine, prazosin, and lisinopril; the proton pump inhibitor pantoprazole; and the beta blocker propranolol.
The medications that were associated with increased suicidal events were reanalyzed, eliminating overdoses to determine if the association was produced by people using them to attempt suicide. Table 3 presents results in which suicide events related to overdose (E950.0-E950.3) were excluded. Only diazepam and promethazine were no longer significant after removing overdoses, but even for these two drugs there remained evidence of increased risk.
OR (95% CIs)
Codeine Phosphate/Promethazine Hydrochloride
Table 4 provides results stratified by age (younger than 18 years versus 18 years and older) and sex for those results that were significantly increased and Table 5 for decreased risk. The increases associated with acetaminophen/hydrocodone and cyclobenzaprine were less pronounced for males. Age and sex stratification yielded consistent results for all of those drugs associated with decreased risk of suicidal events, although the effect of bupropion was somewhat more pronounced in youth (Table 5).
OR (95% CIs)
While there are many interesting associations that deserve further exploration, the decreased risk of suicidal events among patients taking folic acid is particularly interesting. One possibility is that this effect is produced by the use of folic acid during pregnancy; however, Table 5 reveals quite similar effects in men, and also in women ages 18 years and under.
iDEAS provides a high-dimensional approach to signal detection in pharmacoepidemiology providing several major advances over spontaneous reports (FDA MedWatch). iDEAS relies on within-subject comparisons (using well-established existing statistical methods), thereby eliminating between-patient differences in risk factors and many selection effects. Simultaneous screening of hundreds of medications can increase early identification of unexpected effects of newly marketed prescription drugs, which have increased in recent years. Finally, iDEAS identifies both potential risks and benefits, where traditional methods focus on harmful effects, typically one drug or class at a time.
Despite warning labels (FDA, 2008; Hammad, 2004; Stone, 2009) of risk of suicidal ideation or behavior for many drugs used to treat mental health disorders (e.g., antidepressants and anticonvulsants), these drugs were strikingly overrepresented in the list of drugs associated with significantly decreased risk of suicidal events across age and sex groups. The largest decrease was for mirtazapine, which, if confirmed, would be an antidepressant particularly well suited for suicidal patients. Mirtazapine augments both serotonin and noradrenergic transmission similar to trazodone that we also found associated with a lower risk for suicidal events. We previously reported that venlafaxine, which also enhances both serotonergic and noradrenergic systems, has an antisuicidal effect related to its antidepressant action (Gibbons, Hur, Brown, Davis, & Mann, 2012).
Among the medications that screened positive for increased risk were the anxiolytics, alprazolam, clonazepam, and diazepam. Given the current epidemic of opioid-related deaths in the United States (CDC, 2017), it is noteworthy that several opioids appear on the list of drugs with an increased risk of suicide events. Our method produced a more focused list of drugs that increase suicide risk in contrast to more widespread black box designations for CNS and other prescription drugs.
Analgesic/narcotic mixtures acetaminophen/butalbital/caffeine, acetaminophen/hydrocodone bitartrate, and codeine/promethazine were associated with increased risk of suicidal events and these mixtures remain in widespread use (e.g., 3,370,935 patients treated with acetaminophen/hydrocodone bitartrate in 2014 in the MarketScan database). Pain severity is related to suicide risk; however, this benefit appears to be offset by opioid-related increases in suicidal risk (Ilgen et al., 2016). A recent pharmacoepidemiologic study indicates that opioid receptor antagonist methadone may reduce the risk of suicide (Molero et al., 2018). The narcotic antihistamine mixture codeine/promethazine is used to treat pain and as a cough suppressant.
Muscle relaxants carisoprodal and cyclobenzaprine hydrochloride have widespread use (1,166,339 treated with cyclobenzaprine hydrochloride in the MarketScan database in 2014), but there is little attention to their potential for increased suicide risk.
The positive associations between suicidal events and prednisone, the antihistamine/antinausea drug promethazine hydrochloride, and the antibiotics like azithromycin have been unknown and are not on the radar screen of clinicians prescribing these drugs. While it is unlikely that antibiotics are causally linked to suicidal events, they do alter the microbiome, which may affect mental state and suicide risk, and are given when infections are at their worst, a condition that has been shown to be associated with increased suicidal risk (Lund-Sørensen et al., 2016). If confirmed by more detailed longitudinal analyses (Gibbons et al., 2014; Robins, Hernan, & Brumback, 2000), better surveillance and follow-up of patients taking these medications for suicide risk should be initiated.
The decreased risk of suicidal events in patients taking folic acid was not predicted. Interestingly, more than half (52%) of the patients receiving prescriptions for folic acid had a diagnosis of pain, and 16% had a mood disorder diagnosis. In terms of medications prescribed within a year of starting folic acid, 31% filled a prescription for methotrexate in the past year and about 60% received anti-inflammatories or analgesics. Only about 8% received antidepressants. Methotrexate is commonly prescribed for rheumatoid arthritis pain, and methotrexate depletes folate (Selhub, Seyoum, Pomfret, & Zeisel , 1991), so folic acid is often prescribed to replenish it. The low folate levels produced by methotrexate may increase suicide risk, which is decreased following folic acid supplementation. In our data, methotrexate was associated with a 22% increase in suicidal events2 although the multiplicity-adjusted confidence interval included 1.0. In addition, prednisone (21%) and hydrocodone (20%) were also commonly prescribed in the year before a folic acid prescription fill. Both of these drugs were associated with increased risk of suicidal events, and folic acid may reverse this increased risk. This association of folic acid with lower suicide risk does not appear to be confounded by pregnancy. Low folate levels may be linked to higher suicide risk by several mechanisms. Folate deficiency predicts limited clinical response to SSRIs (Miller, 2008), and may enhance the effects of antidepressants acting via monoamine neurotransmitter systems by its involvement in methylation pathways in the one-carbon cycle (Fava & Mischoulon, 2009). Folate levels are reportedly low in blood and red cells in future suicide decedents (Wolfersdorf, Keller, Maier, Fröscher, & Kaschka, 1995), but not in cerebrospinal fluid (CSF) (Engstrom & Traskman-Bendz, 1999). Pan et al. (2017) showed improvement with folic acid treatment in treatment-resistant depression associated with CSF evidence of folate deficiency. These data call for a greater understanding of possible antisuicidal properties of folic acid, potentially leading to innovative, inexpensive prevention opportunities.
The findings of lower than expected suicidal events in patients taking the antihistamine hydroxyzine hydrochloride is of considerable interest, given recent findings that inflammation may be related to suicide risk (Black & Mille, 2015). Antihistamines decrease inflammation and cytokine production, which may lead to decreases in depression and suicidal behavior (Woo et al., 2011).
The finding that use of substance use disorder drugs (buprenorphine/naloxone, disulfiram, and naltrexone) was associated with decreased risk of suicidal events is consistent with the observation that comorbidity of depression and substance use disorders is associated with increased risk of suicide and related behavior (Bukstein et al., 1993; Dhossche, Meloukheia, & Chakravorty, 2000; Tondo et al., 1998). Less alcohol intake is associated with less suicide risk and to the extent these medications reduce alcohol intake, they may reduce suicide risk.
Antihypertensive, alpha1-adrenergic receptor antagonists, prazosin hydrochloride, and trazodone, as well as many antipsychotics and monoamine oxidase inhibitors that also block the alpha1-adrenergic receptor, were associated with lower suicidal event rates. By blocking brain noradrenergic transmission, these medications may reduce noradrenergic drive.
The Parkinson’s disease medication benztropine mesylate is frequently prescribed to reduce or prevent the Parkinsonian side effects of antipsychotic medications that are used for schizophrenia, and also as adjunctive treatment of more severe major depression with standard antidepressants or used conjointly with antidepressants in psychotic depression. Further study of this drug with and without concomitant antipsychotic/antidepressant treatment is indicated.
We found that the anxiolytics alprazolam and diazepam were associated with increased suicidal behavior, however, buspirone was associated with decreased suicidal behavior. Further study is indicated to determine if buspirone is a safer alternative to other anxiolytics.
There are several limitations of our study. First, we rely on ICD-9 codes to determine suicidal events, which are likely to provide underestimates of the true number of patients that actually had a suicidal event. To emphasize this point, we note that a recent study of postpartum women demonstrated that detection of suicidal behavior through natural language processing of clinical notes resulted in an 11-fold greater estimate of suicide attempts than in those based solely on diagnostic data (Zhong et al., 2019). While this underestimate most assuredly affects an estimate of the incidence and possibly the risk difference, it is unlikely to affect the relative risk of suicidal events as reported here. Second, adjustment for dosing, administration schedule, and form are beyond our scope and should be pursued in hypothesis-testing designs. Third, our data on suicide mortality are limited to those patients that died in the hospital or emergency department where a medical claim was submitted. These medical claims data are deidentified and are not linkable to the National Death Index. However, the rate of completed suicides that are not associated with a medical claim is a small fraction of the rate of fatal and nonfatal suicidal events that generate a medical claim, so our estimates should not be greatly affected by the omission of these completions and is consistent with most other pharmacoepidemiologic studies of suicide. Fourth, for drugs used to treat conditions related to suicide such as antidepressants, a suicidal event may lead to treatment, thereby artificially decreasing the postprescription suicidal event rate. Any potentially protective effects identified using this screening technique for such medications should be further studied in more complete longitudinal data using methods that can support causal inferences. We have previously conducted such an analysis using marginal structural models to investigate the dynamic association between antidepressants and suicidal events in youth (Gibbons et al., 2014). We note that eliminating cases in which the prescription and the adverse event occurred on the same day eliminates this effect at least to some degree. Also note that the majority of medications that we have studied here do not suffer from this problem.
We have conducted 922 statistical comparisons with an experiment-wise type I error rate of 5%, which is spread evenly across the 922 comparisons. Despite the large sample size and large number of comparisons, fewer than 6% of the comparisons were statistically significant following adjustment for multiplicity. We note that reliance on statistical significance has been the focus of considerable debate in the statistical literature (Wasserstein & Lazar, 2016) and certainly should not be the only criterion for examining associations in observational data. We have also relied upon ORs, which for the rare event studied here are equivalent to relative risks (RRs). Nevertheless, ORs and RRs have also been criticized as effect size measures (Kraemer, 2004) because they do not provide a measure that can be easily incorporated into a risk-benefit determination. Alternative measures such as number needed to treat (NNT) or number needed to harm (NNH) provide additional information regarding benefits and harms as a function of the number of patients treated. A drug with a large OR or RR may require treatment of a very large number of patients to produce a single additional suicide event, yet may provide benefits that outweigh the risk. For example, using the data in Tables 1 and 2, the NNH for one additional suicide event for alprazolam is 5,713 (95% CI interval = 6,550, 5,067) and the NNT for one fewer suicide event for folic acid is 3,089 (95% CI = 3,820, 2,593). These are large because the suicidal events are rare. In these data, the preexposure rate for alprazolam is 1 in 4,559 and for folic acid it is 1 in 1,990. The differences in preexposure rates between these two medications is likely due to differences in the characteristics of individuals that ultimately fill prescriptions for these two medications. The NNT and NNH are useful for weighing the benefits and risks of these medications, whereas the ORs are useful for signal detection that leads to further investigation.
There are many different approaches to statistical drug surveillance and some of the more recent approaches also are not limited to spontaneous reports. No one method, including ours, is ideal for all applications. A fruitful area for statistical work in drug safety involves the self-controlled case-series design and related statistical methods (Farrington, 1995; Whitaker, Farrington, Spiessens, & Musonda, 2006). Simpson et al. (2013) extended this idea to the multivariable case to explore high-dimensional confounding using a regularized regression framework. This methodology is particularly useful for the study of complex drug interactions, where the regularized regression is used to identify drugs and drug interactions in need of further study. A promising pharmacovigilance method is the TreeScan method (Kulldorff, Fang, & Walsh, 2003; Kulldorff et al., 2013), which uses substantive knowledge to create a hierarchical structure of drugs and/or adverse events into discrete clusters. When such information is available, the TreeScan method provides an interesting alternative approach to dimension reduction under the assumption that medications or adverse events within clusters behave identically. Our approach differs from these methods in that it does not require dimension reduction, but rather estimates all medication (or adverse event) specific effects as EB deviations from the overall model. It is also useful in that it directly provides estimates of both positive and negative associations between the medication exposure and the adverse event(s) of interest. The methods are different and it is an empirical question as to which may be better for a specific application.
In summary, we have used existing generalized mixed-effects regression models to develop a new strategy for drug safety surveillance that is capable of identifying signals for potentially harmful and protective associations for all individual drugs and one or more adverse events. We have illustrated its use on the problem of the association between prescription drugs and suicidal events, one of our nation’s growing epidemics. The application has identified signals for both harmful (e.g., alprazolam) and potentially beneficial (e.g., folic acid) associations that have not been previously considered and, if confirmed in independent hypothesis testing studies, could lead to decreases in the suicide epidemic faced by our country and across the world.
This research was supported by NIH grant R01 MH080122, AHRQ (CERT) grant U18HS016973, and the United States (US) Department of Veterans Affairs Center of Excellence for Suicide Prevention and Office of Mental Health and Suicide Prevention. The contents do not represent the views of the US Department of Veterans Affairs or the United States Government. Correspondence concerning this article should be sent to Robert D. Gibbons, Center for Health Statistics, University of Chicago, 5841 S. Maryland Ave., Room W260, MC2000 | Chicago, IL 60637, Office: 773-834-8692 | Email: firstname.lastname@example.org
All data used in this study were obtained from Truven Health as a part of their MarketScan database under license to the University of Chicago. The study was supported by NIH grant R01 MH080122, AHRQ (CERT) grant U18HS016973. Robert Gibbons has served as an expert witness in cases related to suicide for the US Department of Justice, and Pfizer, Wyeth, and GSK pharmaceutical companies and founded the company Adaptive Testing Technologies that distributes computerized adaptive mental health tests. These activities have been reviewed and approved by the University of Chicago in accordance with its conflict of interest policies. This study was reviewed and deemed exempt from review by the University of Chicago institutional review board.
Au, T. (2016). Getting the most from our safety surveillance: Prescribing information for consideration—A list of medications carrying a suicidality warning. Medication Safety in Seconds, 9(6), 3. https://www.pbm.va.gov/PBM/vacenterformedicationsafety/newsletter/Medication_Safety_in_Seconds_Oct_2016_FINAL.pdf
Bachmann, S. (2018). Epidemiology of suicide and the psychiatric perspective. International Journal of Environmental Research and Public Health, 15(7), 1425. https://doi.org/10.3390/ijerph15071425
Black, C., & Mille, B. J. (2015). Meta-analysis of cytokines and chemokines in suicidality: Distinguishing suicidal versus non-suicidal patients. Biological Psychiatry, 78(1), 28–37. https://doi.org/10.1016/j.biopsych.2014.10.014
Bukstein, O. G., Brent, D. A., Perper, J. A., Moritz, G., Baugher, M., & Schweers, J. (1993). Risk factors for completed suicide among adolescents with a lifetime history of substance abuse: A case-control study. Acta Psychiatrica Scandinavica, 88(6), 403–408. https://doi.org/10.1111/j.1600-0447.1993.tb03481.x
Burke, S. P., Baciu, A., Straton, K., Blumenthal, D., Breckenridge, A., Charo, A., … Gibbons R. D. (2006). The future of drug safety: Promoting and protecting the health of the public. Washington, DC: National Academy Press.
Center for Disease Control (CDC). (2017). Drug overdose deaths. https://www.cdc.gov/drugoverdose/data/statedeaths.html
Center for Disease Control (CDC). (2019). Deaths: Leading causes for 2017. https://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm
Cunningham, F. E., Hur, K., Dong, D., Miller, D. R., Zhang, R., Wei, X., McCarren, M., Mosholder, A. D., Graham, D. J., Aspinall, S. L., & Good, C. B. (2016). A comparison of neuropsychiatric adverse events during early treatment with varenicline or a nicotine patch. Addiction, 111(7), 1283–1292. https://doi.org/10.1111/add.13329
Dhossche, D. M., Meloukheia, A. M., & Chakravorty, S. (2000). The association of suicide attempts and comorbid depression and substance abuse in psychiatric consultation patients. General Hospital Psychiatry, 22(4), 281–288. https://doi.org/10.1016/s0163-8343(00)00085-2
DuMouchel, W. (1999). Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician, 53(3), 177–190. https://doi.org/10.2307/2686093
Engström, G., & Träskman-Bendz, L. (1999). Blood folate, vitamin B12, and their relationships with cerebrospinal fluid monoamine metabolites, depression, and personality in suicide attempters. Nordic Journal of Psychiatry, 53(2), 131–137. https://doi.org/10.1080/080394899426837
Farrington, C. P. (1995). Relative incidence estimation from case series for vaccine safety evaluation. Biometrics, 51(1), 228–235.
Fava, M., & Mischoulon, D. (2009). Folate in depression: Efficacy, safety, differences in formulations, and clinical issues. Journal of Clinical Psychiatry, 70(Supplement 5), 12–17. https://doi.org/10.4088/jcp.8157su1c.03
Gibbons, R. D., & Amatya, A. (2015). Statistical methods for drug safety. London, UK: Chapman and Hall.
Gibbons, R. D., Coca Perraillon, M., Hur, K., Conti, R. M., Valuck, R. J., & Brent D. A. (2014). Antidepressant treatment and suicide attempts and self-inflicted injury in children and adolescents. Pharmacoepidemiology and Drug Safety, 24(2), 208–214. https://doi.org/10.1002/pds.3713
Gibbons, R. D., Hur, K., Brown, C. H., Davis, J. M., & Mann, J. J. (2012). Benefits from antidepressants: Synthesis of 6-week patient-level outcomes from double-blind placebo-controlled randomized trials of fluoxetine and venlafaxine. Archives of General Psychiatry, 69(6), 572–579. https://doi.org/10.1001/archgenpsychiatry.2011.2044
Gibbons, R. D., Hur, K. Brown, C. H., & Mann, J. J. (2009). Relationship between antiepileptic drugs and suicide attempts in patients with bipolar disorder. Archives of General Psychiatry, 66(12), 1354–1360. https://doi.org/10.1001/archgenpsychiatry.2009.159
Gibbons, R. D., & Mann, J. J. (2013). Varenicline, smoking cessation and neuropsychiatric events. American Journal of Psychiatry, 170(12), 1460–1467. https://doi.org/10.1176/appi.ajp.2013.12121599
Gunnell, D., Irvine, D., Wise, L., Davies, C., & Martin, R. M. (2009). Varenicline and suicidal behaviour: A cohort study based on data from the General Practice Research Database. BMJ, 339, Article b3805. https://doi.org/10.1136/bmj.b3805
Hammad, T. A. (2004). Review and evaluation of clinical data. http://www.fda.gov/ohrms/dockets/ac/04/briefing/2004-4065b1-10-TAB08-Hammads-Review.pdf
Hansen, L. (2016). MarketScan white paper: The MarketScan databases for life sciences researchers. Ann Arbor, MI: Truven Health Analytics.
Health Affairs. (2015). The FDA’s Sentinel Initiative. http://www.healthaffairs.org/healthpolicybriefs/brief.php?brief_id=139
Hedeker, D., & Gibbons, R. D. (2006). Longitudinal data analysis. Hoboken, NJ: Wiley. https://doi.org/10.1002/0470036486
Ilgen, M. A., Bohnert, A. S. B., Ganoczy, D., Bair, M. J., McCarthy, J. F., & Blow, F. C. (2016). Opioid dose and risk of suicide. Pain, 157(5), 1079–1084. https://doi.org/10.1097/j.pain.0000000000000484
Kraemer, H. C. (2004). Reconsidering the odds ratio as a measure of 2X2 association in a population. Statistics in Medicine, 23(2), 257–270. https://doi.org/10.1002/sim.1714
Kulldorff, M., Dashevsky, I., Avery, T. R., Chan, A. K., Davis, R. L., Graham, D., … Herrinton, L. J. (2013). Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiology and Drug Safety, 22(5), 517–523. https://doi.org/10.1002/pds.3423
Kulldorff, M., Fang, Z., & Walsh, S. J. (2003). A tree-based scan statistic for database disease surveillance. Biometrics, 59(2), 323–331. https://doi.org/10.1111/1541-0420.00039
Lavigne, J. E., Au, A., Jiang, R., Wang, Y., Good, C. P., Glassman, P., & Cunningham, F. (2012). Utilization of prescription drugs with warnings of suicidal thoughts and behaviors in the USA and the US Department of Veterans Affairs, 2009. Journal of Pharmaceutical Health Services Research, 3(3), 157–163. https://doi.org/10.1111/j.1759-8893.2012.00093.x
Lu, C. Y., Zhang, F., Lakoma, M. D., Madden, J. M., Rusinak, D., Penfold, R. B., . . . Soumerai, S. B. (2014). Changes in antidepressant use by young people and suicidal behavior after FDA warnings and media coverage: Quasi-experimental study. BMJ, 348(7963), Article g3596. https://doi.org/10.1136/bmj.g3596
Lund-Sørensen, H., Benros, M. E., Madsen, T., Sørensen, H. J., Eaton, W. W., Postolache, T. T., . . . Erlangsen, A. (2016). A nationwide cohort study of the association between hospitalization with infection and risk of death by suicide. JAMA Psychiatry, 73(9), 912–919. https://doi.org/10.1001/jamapsychiatry.2016.1594
Mann, J. J., Goodwin, F. R., O'Brien, C. P., & Robinson, D. S.(1993). Suicidal behavior and psychotropic medication: Accepted as a consensus statement by the ACNP Council, March 2, 1992. Neuropsychopharmacology, 8(2), 177–183. https://psycnet.apa.org/doi/10.1038/npp.1993.79
McGilchrist, C. A. (1994). Estimation in generalized mixed models. Journal of the Royal Statistical Society: Series B, 56(1), 61–69. https://doi.org/10.1111/j.2517-6161.1994.tb01959.x
Miller, A. L. (2008). The methylation, neurotransmitter, and antioxidant connections between folate and depression. Alternative Medicine Review, 13(3), 216–226.
Molero, Y., Zetterqvist, J., Binswanger, I. A., Hellner, C., Larsson, H., & Fazel, S. (2018). Medications for alcohol and opioid use disorders and risk of suicidal behavior, accidental overdoses, and crime. American Journal of Psychiatry, 175(10), 970–978. https://doi.org/10.1176/appi.ajp.2018.17101112
National Action Alliance for Suicide Prevention: Research Prioritization Task Force. (2014). A prioritized research agenda for suicide prevention: An action plan to save lives. Rockville, MD: National Institute of Mental Health and the Research Prioritization Task Force.
Pan, L. A., Martin, P., Zimmer, T., Segreti, A. M., Kassiff, S., McKain, B. W., . . . Steinfeld, R. (2017). Neurometabolic disorders: Potentially treatable abnormalities in patients with treatment-refractory depression and suicidal behavior. American Journal of Psychiatry, 174(1), 42–50. https://doi.org/10.1176/appi.ajp.2016.15111500
Robins, J. M., Hernan, M. A., & Brumback, B. (2000). Marginal structural models and causal inference in epidemiology. Epidemiology, 11(5), 550–560. https://doi.org/10.1097/00001648-200009000-00011
Selhub, J., Seyoum, E., Pomfret, E. A., & Zeisel, S. H. (1991). Effects of choline deficiency and methotrexate treatment upon liver folate content and distribution. Cancer Research, 51(1), 16–21.
Simpson, S. E., Madigan, D., Zorych, I., Schuemie, M. J., Ryan, P. B., & Suchard., M. A. (2013). Multiple self-controlled case series for large-scale longitudinal observational databases. Biometrics, 69(4), 893–902. https://doi.org/10.1111/biom.12078
Stroud, A. H., & Secrest, D. (1966). Gaussian quadrature formulas. Englewood Cliffs, NJ: Prentice-Hall.
Stone M., Laughren T., Jones M. L., Levenson M., Holland P. C., Hughes A., Hammad T. A., & Temple R. (2009). Risk of suicidality in clinical trials of antidepressants in adults: analysis of proprietary data submitted to US Food and Drug Administration BMJ, 339, Article b2880. https://doi.org/10.1136/bmj.b2880
Tippett, E. C., & Chen, B. K. (2015). Association of attorney advertising and FDA action with prescription claims: A time series segmented regression analysis. Drug Safety, 38(12), 1169–1178. https://doi.org/10.1007/s40264-015-0340-6
Tondo, L., Baldessarini, R. J., Floris, G., Silvetti, F., Hennen, J., & Tohen, M. (1998). Lithium treatment and risk of suicidal behavior in bipolar disorder patients. Journal of Clinical Psychiatry, 59(8), 405–414. https://doi.org/10.4088/jcp.v59n0802
U.S. Food and Drug Administration (FDA). (2008). Statistical review and evaluation: Antiepileptic drugs and suicidality. https://www.fda.gov/files/drugs/published/Statistical-Review-and-Evaluation--Antiepileptic-Drugs-and-Suicidality.pdf
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on p-values: Context, process, and purpose. The American Statistician. 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108
Whitaker, H. J., Farrington, C. P., Spiessens, B., & Musonda, P. (2006). Tutorial in biostatistics: The self-controlled case series method. Statistics in Medicine, 25(10), 1768–1797. https://doi.org/10.1002/sim.2302
Wolfersdorf, M., Keller, F., Maier, V., Fröscher, W., & Kaschka, W. P. (1995). Red-cell and serum folate levels in depressed inpatients who commit violent suicide: A comparison with control groups. Pharmacopsychiatry, 28(3), 77–79. https://doi.org/10.1055/s-2007-979594
Woo, J. M., Gibbons, R. D., Qin, P., Kamarow, H., Kim, J. B., Rogers, C. A., … Postolache, T. T. (2011). Suicide and prescription rates of intranasal corticosteroids and non-sedating antihistamines for allergic rhinitis: An ecological study. Journal of Clinical Psychiatry, 72(10), 1423–1428. https://doi.org/10.4088/jcp.10m06765
Wysowski, D. K., & Swartz, L. (2005). Adverse drug event surveillance and drug withdrawals in the United States, 1969–2002—The importance of reporting suspected reactions. Archives of Internal Medicine, 165(12), 1363–1369. https://doi.org/10.1001/archinte.165.12.1363
Zhong, Q. Y., Mittal, L. P., Nathan, M. D., Brown, K. M., Knudson González, D., Cai, T., … Williams, M. A. (2019). Use of natural language processing in electronic medical records to identify pregnant women with suicidal behavior: Towards a solution to the complex classification problem. European Journal of Epidemiology, 34(2), 153–162. http://doi.org/10.1007/s10654-018-0470-0
View “All 922 Medications Included in the Analysis” in this supplementary file.
©2019 Robert Gibbons, Kwan Hur, Jill Lavigne, Jiebiao Wang, and J. John Mann. This article is licensed under a Creative Commons Attribution (CC BY 4.0) International license, except where otherwise indicated with respect to particular material included in the article.