This study (n=158) examined the psychometric properties of three commonly used rating scales (MEQ, HRS, ARCI). The authors found only sparing agreement of their psychometric analyses with the theoretically proposed models of the scales.
Abstract
“Objective: In the present study we explored the psychometric properties of three widely used questionnaires to assess the subjective effects of hallucinogens: the Hallucinogen Rating Scale (HRS), the Mystical Experience Questionnaire (MEQ), and the Addiction Research Center Inventory (ARCI). Methods: These three questionnaires were administered to a sample of 158 subjects (100 men) after taking ayahuasca, a hallucinogen whose main active component is N,N‐dimethyltryptamine (DMT). A confirmatory factorial study was conducted to check the adjustment of previous data obtained via theoretical proposals. When this was not possible, we used an exploratory factor analysis without restrictions, based on tetrachoric and polychoric matrices and correlations. Results: Our results sparsely match the theoretical proposals of the authors, perhaps because previous studies have not always employed psychometric methods appropriate to the data obtained. However, these data should be considered preliminary, pending larger samples to confirm or reject the proposed structures obtained. Conclusions: It is crucial that instruments of sufficiently precise measurement are utilized to make sense of the information obtained in the study of the subjective effects of psychedelic drugs. Copyright © 2016 John Wiley & Sons, Ltd.”
Authors: José Carlos Bouso, Eduardo José Pedrero‐Pérez, Sam Gandy & Miguel Ángel Alcázar‐Córcoles
Summary
Classic hallucinogens such as Lysergic acid Diethylamide (LSD), mescaline, psilocybin, or DMT induce cortical blood flow activation in frontal and paralymbic areas of the brain, as well as decrease in the metabolic activity of the default mode network.
Contemporary psychedelic research employs different rating scales to assess the subjective effects of research subjects and/or patients. However, fewer efforts have been made to improve the psychometric properties of the different rating scales. Among the rating scales, OAV and MEQ are the only ones that have been subject to rigorous psychometric analysis. However, the ARCI and HRS factorial structure and internal consistencies (reliability) have been poorly analyzed with statistical methods that have already been surpassed.
The MEQ is a modified version of the State of Consciousness Questionnaire (SCQ), a psychological rating scale originally constructed by Pahnke to assess the spiritual peak effects of hallucinogens. It has 100 items, 43 of which correspond to the six subscales of the SCQ. The SCQ has been utilized in studies exploring the mysticomimetic properties of psilocybin, ayahuasca, DMT, dextromethorphan, and the psychotherapeutic potential of psilocybin and LSD. MacLean et al. (2012) and Barrett et al. (2015) explored the validity, reliability and factor structure of SCQ, which was composed of 30 final items explaining 57% of the variance. A confirmatory factorial analysis was performed on 440 subjects, and a 30-item questionnaire was created. The questionnaire was confirmed to have a four-factor structure in a series of clinical studies using psilocybin.
The authors defined six conceptually coherent clusters and performed a principal components factor analysis after administering four different doses of DMT and one of placebo to 11 subjects. They did not report either data regarding the items load for each factor nor indices of internal consistence. Riba et al. (2001) translated the version 3.06 of HRS to Spanish and administered it to 127 subjects in two different studies. They found that Factor 1 was composed by Somaesthesia, Perception, Cognition, Affect and Intensity, and Factor 2 was composed by Volition. The HRS has been extensively used in hallucinogen research after the administration of psilocybin, DMT, ketamine, Salvia divinorum, 2C-B, MDE, MDMA, and ayahuasca, and in psychotherapeutic studies with psilocybin and MDMA.
The ARCI is a 600-item questionnaire developed for the assessment of subjective effects and abuse potential of psychoactive drugs. It has been used to assess the effects of an extensive number of drugs, including cocaine, GHB, benzodiacepines, opiates, piperazines and amphetamines. The ARCI consists of five dimensions: MBG (morphine-benzedrine group), PCAG (pentobarbital-chlorpromazine-alcohol group), LSD (lysergic acid diethylamide scale), BG (benzedrine group), and A scale (amphetamine). The final number of items is different for each subscale. Lamas et al. (1994a) developed a Spanish version of the ARCI and obtained good reliability indices for it.
A short version of the ARCI was developed, and the reliability indices were higher than those obtained by Haertzen (1974) for the original version, but it is not possible to establish direct comparisons because the number of items composing each subscale is different in each version.
Psychometric questionnaires to assess subjective effects of hallucinogens have numerous limitations that make the interpretation of results difficult. In this study, we present novel psychometric data for the MEQ, HRS, and ARCI questionnaires and propose preliminary new versions and/or alternative interpretations of the data for each of them.
Participants
Researchers contacted ayahuasca practitioners in Spain, who provided information about when the ceremonies were taking place. Once present, researchers asked the participants if they were willing to participate in the study.
Instruments
The MEQ is a 30-item self-report questionnaire designed to measure single mystical experiences that result from ingesting hallucinogens. It was translated into Spanish according to the principles of back translation, and placing the emphasis on conceptual rather than linguistic equivalence.
In this study we used the Spanish version of the HRS, which consists of 71 Likert-type items scoring from 0 to 4.
The ARCI is a self-report questionnaire that contains five group variability scales: MBG, PCAG, LSD, BG, and A. The Spanish version was adapted by Lamas et al. (1994a).
Data analysis
We performed a confirmatory factorial analysis using the AMOS 18 software, using the Unweighted Least Squares method when multivariate normality could not be guaranteed. An Unrestricted Exploratory Factor Analysis was performed when the theoretical structure could not be empirically replicated.
We performed an optimized parallel analysis based on minimum rank factor analysis, then an overall factor analysis, using a Simplimax rotation and several criteria to guarantee the simplicity of the studied solution. The internal consistency of each scale was estimated using the Cronbach’s alpha.
Sample
The questionnaires were completed by 167 subjects after the ayahuasca ceremony, and 134 had previous experience with psychedelics (mean of 21 times with a range from 0 to 350). The mean intensity of the perceived dose was a medium dose.
We carried out a confirmatory factorial analysis of the tetrafactorial solution proposed by MacLean et al. (2012) on the data obtained and performed an exploratory analysis.
The results obtained after the Mardia’s analysis showed that the data did not comply with multivariate normality (p .001), so we configured a polychoric correlation matrix on which we performed a factor analysis. The two factors explained 59.11% of the common variance.
The Schmid – Leiman transformation revealed a second order factor that links both factors, and the scale as a whole might be considered unidimensional.
We conducted a confirmatory factorial analysis on the solution proposed by Riba et al. (2001) (71 items in 6 factors), but the goodness of fit indicators were far from optimal values. Therefore, we conducted an exploratory analysis of the available data, eliminating 12 items that did not show high-corrected item-test correlation with any scale.
The ARCI questionnaire is compounded by five dimensions, and the derived scales also presented adequate internal consistency (0.75 – 0.90). All of the items displayed good discriminative capacities (>0.30 in all cases), and the theoretical structure of the questionnaire was followed by the exploratory study.
A solution was obtained for the data by using 18 items with significant factorial loadings, but the solution only explained 50.5% of the common variance, and the internal consistency was precarious.
Correlations between the factorial derived subscales
Regarding correlations between different subscales for each specific questionnaire, all the subscales of the MEQ correlate positively between themselves, except for Agitation, which is negatively correlated with Sedations.
The MEQ and HRS correlate with each other, except for AG, which does not correlate with either.
This paper analyzes the factorial structure and internal consistency of three rating scales destined to measure the subjective effects of psychedelics. Only the MEQ has been explored with modern psychometric techniques, while the HRS and ARCI have only been subject to limited psychometric research.
MacLean et al. (2012) explored the factorial structure and reliability of the MEQ, a questionnaire that had different names and versions over time and was extensively used in early psychedelic research.
In a recent study using psilocybin, Barrett et al. (2015) confirmed the structure found by MacLean et al. (2012), and found a two-factor model explaining 59.11% of the common variance, a figure similar to those found by MacLean et al. (2012) and Barrett et al. (2015). The two factors we relabeled as Mystical ecstasy and Transdimensionality had excellent reliability indices and were similar to the factors found by MacLean et al. (2012).
Factor structure differences may be due to sample size, conditions of assessment, subjective effects of substances, and cultural differences between the samples. Small discrepancies between our model and MacLean’s model probably do not lie in eventual low scores yielded by our sample.
Future studies should cover more culturally diverse samples and incorporate a wider range of substances to define the most suitable MEQ factor structure.
The Strassman et al. (1994) HRS was based on 11 subjects and a principal components factor analysis. A confirmatory factor analysis was conducted with HRS items and 6 factors were found, but the percentage of explained variance was lower (56.5%) than that obtained by Riba et al. (2001). Riba et al. (2001) found two subscales with inadequate univariate internal consistency indices, but we found all six subscales to have good or excellent reliability parameters. The multivariate reliability indices for the whole scale were excellent.
We considered our model as more appropriate to analyze the HRS scores, and relabeled the 6 factors as Sensorial distortion, Cognitive distortion, Agitation, Security/Control, Visual distortion, and Quality of the experience.
The ARCI 49-item short form was found to have a factorial structure, and a new 18-item solution with three factors was found. The reliability indices for the whole test proved to be precarious, but the new factor names were relabeled. The original version of the ARCI questionnaire is composed of items different from those informing the subscales, so it is not possible to establish direct comparisons between their internal consistency indices. However, the 18-item version has a low percentage of variance and adequate multivariate indices.
The correlation analysis between the different subscales of the MEQ showed that the Mystical Ecstasy subscale did not correlate with Agitation, Activation and Sedation, but did correlate with Euphoria. Therefore, the MEQ may be a good map of the mystical experience achieved by our subjects under the effects of ayahuasca.
The ARCI showed a single significant positive correlation between subscales EUP and ACT, a negative correlation between subscales ACT and SED, and no significant correlation between subscales EUP and SED.
The positive and significant correlations between the MEQ subscales and the HRS subscales may be reflecting two very dependent aspects of the psychedelic experience, the psychedelic one and the mystical one.
Although our study was carried out in subjects following an ayahuasca experience, larger samples are required to explore if explained common variances at the different rating scales could be improved. Moreover, future studies should incorporate more substances aiming to confirm or reject the factorial structures we found.
The results emerging from some studies using the three rating scales explored in our study should be interpreted with caution. Future studies should take into account the strong limitations of the mentioned rating scales whenever they plan to use them. The HRS and ARCI questionnaires could be improved by proposing new tentative items, and all rating scales destined to be used in psychopharmacology research could be psychometrically analyzed according to these new statistical methods.
In Psychopharmacology, and more specifically in the case of Psychopharmacology of psychedelic drugs, it is common to publish results based on poorly psychometrically explored questionnaires. Despite this, the three questionnaires discussed here have been demonstrated to be sensitive for discriminating subjective effects among different substances.
Study details
Participants
158