- Research Article
- Open access
- Published:
Comparison of Point Placement by Veterinary Professionals with Different Levels of Acupuncture Training in a Canine Cadaver Model
Innovations in Acupuncture and Medicine volume 10, pages 360–370 (2017)
Abstract
Veterinary acupuncture is becoming increasingly implemented for various disease processes, with growing numbers of veterinarians pursuing advanced training to meet the rising demand for this relatively new intervention. Accurate acupoint placement remains challenging, with individual practitioners relying on varying methods of point identification, often compounded by the transpositional nature of points for companion animals. The aim of this study was to assess for differences in acupuncture needle placement of select points between veterinary professionals with three different levels of acupuncture training in an academic teaching environment. Seven participants placed a total of six acupoints on a canine cadaver. Digital radiography was used to document each participant’s point placement. Each participant’s point location was then compared to a control “correct” point, and the distance between the two points was measured. A significant difference in placement accuracy was identified between the participants when grouped by training level (p = 0.03). These results indicate that veterinary patients receiving acupuncture treatment from veterinarians with different levels of training may subsequently experience varying effects, although further studies are warranted on more specific acupoint description as well as the clinical implications of needle placement accuracy.
1. Introduction
Acupuncture is increasingly administered in human and veterinary medicine as a primary or adjunctive integrative therapy for various clinical conditions [1–4]. The Canon of Veterinary Medicine, purportedly ascribed to Chinese authors in 620 B.C.E., is regarded as the first description of techniques representing possible early forms of acupuncture, but persistent controversies exist regarding the origins of the practice in Chinese medical history [5–7]. This and other historical Chinese texts largely address equine acupuncture. Acupuncture for companion animals is a more modern development, with significant contributions and growth in American and European veterinary communities in the past four decades [8]. Veterinary acupuncture across species has been evaluated in a number of clinical conditions and although older meta-analyses failed to support a definitive effect from the treatments, subsequent clinical trials investigated its role in pain control, neurologic disease, gastrointestinal disease, emergency resuscitation, behavior, dermatologic disease, and reproduction [9,10].
The American Veterinary Medical Association (AVMA) recognizes veterinary acupuncture as a subset of complementary and alternative veterinary medicine (CAVM), which are considered part of the practice of veterinary medicine [11]. Most American states require that veterinary acupuncture be practiced by a licensed veterinarian, but some laws possess different provisions regarding the scope of practice. No regulatory body oversees the training of veterinary acupuncturists. Courses are available from continuing education purveyors, including the Chi Institute, Colorado State University, and the International Veterinary Acupuncture Society. No studies have critically evaluated competencies or outcome measures as a function of training program or experience.
Veterinary exposure and training in the field of veterinary acupuncture remains variable. A survey of AVMA-accredited veterinary schools in 2000 documented a perceived lack of educational or research programs in CAVM within the veterinary profession [12]. A follow-up survey in 2011 identified an increase in CAVM coursework, much of it inclusive of acupuncture, from approximately 30% to 47% of programs, with a general consensus that further structured curricula was warranted [13]. Recent educational guidelines for CAVM, increasingly identified as integrative medicine when combined with other techniques, suggested that students, at minimum, receive an opportunity to develop baseline knowledge of the more commonly applied modalities of CAVM such as acupuncture in order to meet the increasing public demand for veterinary expertise in these topics [14]. Institutions that adopt this model also often have a high clinical caseload in the area; a 2015 retrospective study of 5195 integrative patient treatments at one veterinary academic teaching hospital demonstrated that acupuncture was incorporated into 81.5% of all treatment sessions [15].
Two systems of veterinary acupuncture point classification exist in veterinary medicine: classical points and transpositional points. Classical points are acupoints that were documented based on experiential effects or associations in China and were often unique to animals, with the most comprehensive set of points described in horses. These points retain their Chinese names and do not have a channel—number system, but are rather identified by anatomic location alone. Transpositional acupoints, those points transposed from human meridian and point anatomy to the structure of treated veterinary species, were developed in the 1970s when Western veterinarians and Chinese human acupuncturists developed an atlas of adapted acupoints [16]. Anatomic differences posed challenges for the transposition of some points. For example, the tail does not have an analogous structure on a human and an equine distal limb has a single digit. The placement of LI-4 in dogs remains complicated by the vestigial nature of the first digit as opposed to the opposable thumb in humans; consequently, this point may variably be placed on the medial surface of the second metacarpal bone or in the space between the second and third metacarpals. The dog also lacks a functional soleus muscle, with possible implications for placement and utility of SP-6. There are ongoing investigations into the neuroanatomic location of acupuncture points in veterinary species, and the “correct” locations and dimensions of acupoints, in both humans and animals, remains a subject of debate [16–18].
The relative placement of acupuncture points is guided by palpation and a proportional unit of measurement, known in Chinese as a “cun”. This is defined as the width of the interphalangeal joint at a human patient’s thumb, whereas it has been described as the width of a domestic animal’s last rib [19]. The width and/or length of other \ body areas, such as the antebrachium or tibia, have been defined as a set number of cun; the length of the antebrachium, for example, is 12 cun from the elbow crease to the carpal crease (Fig. 1). Veterinary acupuncturists use these guidelines as well as direct palpation to locate specific acupoints.
A scientific model of acupuncture effects relies on accurate needle placement for appropriate stimulation of neural, fascial, vascular, and other structures that may be small to microscopic. Therefore, variability in needle placement could profoundly influence clinical effects. The interindividual variability of needle placement by veterinary acupuncture practitioners has not been assessed. The goal of this study was to evaluate differences in acupuncture needle placement in selected points between veterinary professionals with three different levels of acupuncture training in an academic teaching environment. It was hypothesized that more training would result in placement nearer the true acupuncture point, as defined by consensus between all participants while consulting point diagrams and established anatomic descriptions [19].
2. Materials and methods
2.1. Specimens
One adult medium-sized canine cadaver euthanized for reasons unrelated to this study was obtained from a local animal shelter. The dog was stored in a freezer at −20°C for 48 hours and then moved to a refrigerator at 2°C for 24 hours to allow for adequate thawing. After completion of this study, the cadaver was further used for training of surgical residents on orthopedic procedures.
2.2. Study participants
Seven veterinary professionals with different levels of training in veterinary acupuncture participated in the study. Two were licensed veterinarians who are also certified in veterinary acupuncture and currently hold faculty positions in the Integrative Medicine service of an academic teaching hospital. Three were house officers (2 interns, 1 1st-year resident) in their first 3 months of training in the same Integrative Medicine service with exposure to acupuncture and traditional Chinese veterinary medicine as well as physical rehabilitation, nutrition, and hyperbaric oxygen therapy. Two participants were senior veterinary students who completed 4 weeks of elective experiential training in the service under the direction of the aforementioned faculty and house officers. This service administers treatments to approximately 20 small animal patients daily, with more than 80% of those receiving acupuncture [15].
2.3. Equipment
Gold semipermanent ear needles (Fig. 2) were used for point placement. Digital radiographs were acquired after all needle placements using a Sedecal 50-kW generator, Toshiba X-ray tube (Toshiba Electron Tubes & Devices Co., LTD.; Otawara, Japan), and a 50G Canon digital plate. All images were stored and analyzed using picture archiving and communication system (PACS).
2.4. Experimental procedure
Needle placement was performed in the unshaved cadaver in a blinded fashion by each veterinary professional, and digital radiography was performed. Each study participant placed a total of six acupoints (Table 1, Figs. 3 and 4). These acupoints were selected for analysis based on their common use and application in veterinary acupuncture.
Diagram of canine acupoints with LI-10 (solid white arrow) and PC-6 (solid yellow arrow) highlighted. Note. From H. Xie, Veterinary Acupuncture Atlas, Chi Institute, Reddick, FL, USA, 2003. With permission. GB = gallbladder; ST = stomach; LU = lung; PC = pericardium; LI = large intestine; TH = triple heater; SI = small intestine; HT = heart; LV = liver; SP = spleen.
The cadaver was secured to the imaging table using medical tape for each round of acupoint placement and was not manipulated from such a position during each placement. An external scaling marker was secured in the same plane as the anatomical location of the acupoint being assessed. For example, the marker was secured at the level of the greater trochanter for GB-30 evaluation. A preplacement image was acquired prior to any needle placement. One study member entered the imaging suite and placed a needle in the cadaver at the acupoint being assessed. All other individuals were located outside of the suite during placement with no visual access to the cadaver. A digital radiograph was obtained with the needle in place. The needle was then removed by the member who placed it, leaving no visible indicators of where it had been located and a clean surface for the next member to place their needle. Once all participants had placed one acupoint, all members gathered to place a control needle in the location based on group consensus with reference to the point’s anatomical description and published resources [19]. This control needle was accepted as the correct location for the purposes of the study. Once an image of the control needle was obtained, the cadaver was then moved into the next position for subsequent point placements, and the process was repeated. The cadaver was secured in left lateral recumbency for PC-6, LI-10, and GB-30 with the left thoracic limb used for PC-6 application, the right thoracic limb for LI-10, and the right pelvic limb for GB-30. The cadaver was secured in sternal recumbency for GV-20, BL-23, and Bai Hui with the head and cervical spine parallel to the table.
The primary author (TY) performed digital measurements through PACS after all images had been collected. Each image was adjusted for magnification prior to any measurements. Two reference points were assigned above and below each acupoint location at visible anatomical landmarks to create a linear axis; these were kept consistent for each set of images. Measurements were obtained between the control locations and participants’ placements of each point to this constant reference line (Fig. 5). Scatter plots were created to display the results, assigning the consensus control location to the origin and adjusting members’ attempted locations accordingly in a Cartesian plane (Figs. 6–11). Published anatomic descriptions were used to establish a “correct” radiographic location by identifying landmarks and appropriate proportional cun measures for each acupoint (Fig. 1). More specifically, digital measurement of the width of the last rib and subdividing the distance between two anatomical landmarks according to historical precedent established the appropriate cun measure for each point. The distance between each participant’s placement and the radiographic point identification was then calculated as described above.
The veterinary professional with the largest distance between their placement and the correct (anatomical and radiographic) location was identified. The remaining members’ distances were then compared to this value to create ratios for comparison and analysis.
2.5. Statistical analysis
Data were analyzed using commercially available statistical software (Minitab Express 1.5.0, 2016; Minitab Inc., State College, PA, USA). Data were assessed for normality using a Shapiro—Wilk test. One-way analysis of variance was performed on normal data to compare the sets of measurements for each group sorted by training level. Nonparametric data were compared with Kruskal—Wallis to evaluate the difference in the placement deviations between points. Statistical significance was established if the probability of error was less than 5%.
3. Results
All seven study participants placed a needle at each of six acupoints. Participant data were sorted to three different groups according to the level of veterinary acupuncture training: faculty (n = 2), house officers (n = 3), and students (n = 2).
3.1. Descriptive statistics
The mean placement deviation from the accepted anatomic points across all groups was 16.7 mm, with a range of 1.8–40.5 mm. The deviation of the placements when assessed according to the radiographically determined control point was 24.8 mm, with a range of 3.98–67.6 mm. The deviations according to each acupoint are shown in Table 2.
3.2. Study members grouped by training level
A statistically significant difference was identified in the participant groups established by training level (p = 0.03) for data using the anatomically defined control point. Post hoc analysis identified faculty acupoint placements were significantly more accurate compared to both house officers (0.32 ± 0.18 vs. 0.56 ± 0.31, p = 0.01) and students (0.32 ± 0.18 vs. 0.62 ± 0.34, p = 0.015). A difference was not detected between house officer acupoint placements and students (p = 0.61). No difference was detected between groups when data were used with a control point established radiographically (p = 0.15).
3.3. Individual acupoint deviations
A statistically significant difference was identified among individual acupoints for deviations from the radiographically defined control points (p = 0.0002; Table 2). Post hoc analysis identified significant differences in placement deviations between GV-20 and BL-23 (p = 0.0215), GV-20 and Bai Hui (p = 0.0215), GV-20 and GB-30 (p = 0.0215), GV-20 and PC-6 (p = 0.0073), LI-10 and BL-23 (p = 0.0033), LI-10 and Bai Hui (p = 0.0022), LI-10 and GB-30 (0.0033), and LI-10 and PC-6 (p = 0.0033). A difference was not detected between GV-20 and LI-10 (p = 0.0967). No difference was detected between individual acupoints when comparing deviations for the anatomically defined control points (p = 0.0716).
4. Discussion
The results of this study revealed a statistically significant difference in the accuracy of acupuncture needle placement when veterinary acupuncturists were grouped by training level. Specifically, faculty members who were certified in veterinary acupuncture and possessed more clinical experience placed needles with greater accuracy as compared to less-experienced house officers and students. However, the greatest faculty deviation from a single accepted point was up to 22 mm and up to 67 mm of combined error across all seven points for an individual. Similar inconsistencies were noted in a human study assessing needle placement using two different methods of point location, with the range of the distance between markings made by participants attempting the same acupoint measuring more than 60 mm [20]. This level of inaccuracy raises potentially serious implications regarding the practice of acupuncture in clinical application, formal training, and research. The possibility of not stimulating the functional biologic determinants of an acupoint, for example, neural structures, or placing a needle into a different acupoint than intended have implications for therapeutic efficacy and safety, as well as repetitive reliability. These results must, however, be interpreted in light of the uncertainty regarding the exact histological features that characterize acupoints and connecting meridians [17,21]. Until such time as there is clarity on the structural aspects of each acupuncture point with documented evidence of clinical efficacy upon stimulation, the ability to define and designate specific dimensions of a fixed, correct acupoint location remain uncertain. The transpositional nature of small animal acupuncture points further magnifies such concerns as the experiential basis for such points is substantially reduced in these species as compared to humans. Moreover, some practitioners rely heavily on palpation and believe that points are not fixed across patients but rather defined by anatomic variations, unique structure, and response to needle stimulation [18,22–25].
Multiple human studies identified similar results when needles were placed in sham acupuncture locations not accepted as defined acupoints [26]. A randomized, controlled trial evaluating chronic knee pain in humans found that both acupuncture and sham acupuncture groups demonstrated double the improvement compared to the conventional medical treatment group [27]. Conversely, a systematic review in humans found that stimulation of PC-6 resulted in less postoperative nausea and vomiting and a reduced need for rescue antiemetics, as compared to sham treatment [28]. A study in dogs showed that stimulation of PC-6 alone and in combination with four other acupoints after morphine administration yielded a significantly lower total number of vomiting and retching events compared to saline solution or sham acupuncture [29]. Any placebo effects of sham treatment that may exist have implications for the study findings herein. If the point location is not dependent on stimulation of small histologic structures, then the accuracy of point placement is potentially of lesser import. Additional study in dogs is necessary to further refine the clinical importance of needle placement accuracy in patient outcomes.
This study suggests that patients receiving acupuncture treatment from individuals with different levels of training could have differential effects, specifically at the academic institution at which this study was conducted. However, anecdotal clinical observation suggests that the majority of patients respond positively to acupuncture therapy regardless of the skill level of the practitioner, although this has not been scientifically verified. This could support a sham effect or a broader connection between acupuncture points across anatomic locations. Alternatively, other interventions may influence outcome to a greater degree than acupuncture or have a synergistic effect. The service in which the participants attend to patients incorporates a number of concurrent additional therapeutic modalities such as therapeutic ultrasound, laser, and rehabilitative exercises [15]. Moreover, the systemic effects of needle placement are likely mediated by endogenous opioids, serotonin, substance P, and other compounds [30], and given that multiple needles are placed in each patient, perhaps only a percentage are required to be accurate to elicit such systemic effects.
The differences between anatomically defined and radiographically defined control points are noteworthy. The differences between training levels were only statistically significant when considering the anatomically determined correct location. For individual acupoints, GV-20 and LI-10 deviations compared to all other points were statistically greater when considering only the radiographically determined correct locations. The clinical relevance of the digitally determined correct radiographic location may be inconsequential when considering the nature of acupuncture. Rather than using standardized measurements across all patients, acupuncturists traditionally use knowledge of meridian/acupoint locations in relation to anatomical landmarks along with direct palpation of tissue to determine where a needle will be placed [23]. Radiographs are by nature two-dimensional images and therefore the influence of positioning, assessment of critical landmarks, and other factors may preclude this method from reliably establishing the location of an acupuncture point.
A significant limitation of this study was the low number of data with only two to three participants per training level. However, expansion of the study would have required inclusion of individuals from outside institutions with different training and clinical approaches, all of which could introduce new variability and bias. Institutional consistency was therefore prioritized over the number of study participants. The low study numbers also required that the data for all points be used for each individual, rather than assessing each acupuncture point for difference between groups. Therefore, significant outliers for one or more acupuncture points could artificially influence statistical findings. The use of a cadaveric model rather than a live dog may also have introduced some disparity from clinical needle placement and prevented an evaluation of physiological response, but assessment of clinical effects was not a specific goal of this study. The advantage of being able to manipulate the cadaver and avoid motion was prioritized above these factors. The manner in which the cadaver was prepared did allow for similar levels of tactile feedback during palpation to place needles.
These data represent the first in veterinary medicine on the accuracy of point placement and serve as a baseline for continued study. Additional investigations are clearly warranted on acupoint specificity, which may be best achieved with a systematic three-dimensional study on each major acupoint and even histological assessment of point structure. Preliminary studies using dissection suggested revision to some transpositional points, which have not yet been adopted by all practitioners [31]. Further analysis of inter-individual needle placement with larger data sets paired with objective outcome measures is required to monitor efficacy, guide development of a more accurate method of point location, and minimize any placebo effect. Additional studies should also investigate intraindividual variability of needle placement across patients given the large variation of anatomy in veterinary patients, even within the same species. Although the complexity and individualized approach of acupuncture methodology poses challenges for conventional study design [32,33], high-quality randomized controlled trials and evidence-based medicine should continue to be aggressively pursued in both human and veterinary acupuncture to determine the specific effects of this increasingly popular veterinary intervention.
References
Ma C, Sivamani RK. Acupuncture as a treatment modality in dermatology: a systematic review. J Altern Complement Med. 2015;21:520–529.
Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, et al. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012;172: 1444–1453.
Lane DM, Hill SA. Effectiveness of combined acupuncture and manual therapy relative to no treatment for canine musculoskeletal pain. Can Vet J. 2016;57:407–414.
Koski M. Acupuncture for zoological companion animals. Vet Clin North Am Exot Anim Pract. 2011;14:141–154.
Jaggar DH, Robinson NG. History of veterinary acupuncture. In: Schoen AM, ed. Veterinary Acupuncture: Ancient Art to Modern Medicine. 2nd ed. St. Louis, MO: Mosby Elsevier; 2001:13.
Ramey DW, Imrie R, Bowles D. Acupuncture founded on magical and mystical musings about how the world works. Can Vet J. 1999;40:840–841.
Imrie RH, Ramey DW, Buell PD, Ernst E, Basser SP. Veterinary acupuncture and historical scholarship: claims for the antiquity of acupuncture. Sci Rev Alt Med. 2001;5:135–141.
Xie H, Ortiz-Umpierre C. What acupuncture can and cannot treat. J Am Anim Hosp Assoc. 2006;42:244–248.
Habacher G, Pittler MH, Ernst E. Effectiveness of acupuncture in veterinary medicine: systematic review. J Vet Intern Med. 2006;20:480–488.
Xie H, Sivula N. Review of veterinary acupuncture clinical trials. Am J Trad Chin Vet Med. 2016;11:49–60.
AVMA. Complementary, alternative, and integrative veterinary medicine. Available at: https://www.avma.org/KB/Policies/Pages/AVMA-Guidelines-for-Complementary-and-Alternative-Veterinary-Medicine.aspx [Date accessed: April 7, 2017].
Schoen AM. Results of a survey on educational and research programs in complementary and alternative veterinary medicine at veterinary medical schools in the United States. J Am Vet Med Assoc. 2000;216:502–509.
Memon MA, Sprunger LK. Survey of colleges and schools of veterinary medicine regarding education in complementary and alternative veterinary medicine. J Am Vet Med Assoc. 2011;239:619–623.
Memon MA, Shmalberg J, Adair III HS, Allweiler S, Bryan JN, Cantwell S, et al. Integrative veterinary medical education and consensus guidelines for an integrative veterinary medicine curriculum within veterinary colleges. Open Vet J. 2016; 6:44–56.
Shmalberg J, Memon MA. A retrospective analysis of 5,195 patient treatment sessions in an integrative veterinary medicine service: patient characteristics, presenting complaints, and therapeutic Interventions. Vet Med Int. 2015. https://doiorg.publicaciones.saludcastillayleon.es/10.1155/2015/983621.
Robinson NG. Veterinary acupuncture: an ancient tradition for modern times. Altern Complement Ther. 2007;13: 259–265.
Ramey DW. A review of the evidence for the existence of acupuncture points and meridians. 46th AAEP Annual Convention, November 26–29, 2000, San Antonio, TX, USA.
Longhurst JC. Defining meridians: a modern basis of understanding. J Acupunct Meridian Stud. 2010;3:67–74.
Xie H, Preast V. Xie’s Veterinary Acupuncture. 1st ed. Ames, IA: Blackwell Publishing; 2007.
Aird M, Coyle D, Cobbin M, Zaslawski C. A study of the comparative accuracy of two methods of locating acupuncture points. Acupunct Med. 2000;18:15–21.
Wang GJ, Ayati MH, Zhang WB. Meridian studies in China: a systematic review. J Acupunct Meridian Stud. 2010;3:1–9.
Kong J, Gollub R, Huang T, Polich G, Napadow V, Hui K. Acupuncture De Qi, from qualitative history to quantitative measurement. J Altern Complement Med. 2007;13:1059–1070.
Finando S, Finando D. Qi, acupuncture, and the fascia: a reconsideration of the fundamental principles of acupuncture. J Altern Complement Med. 2012;18:880–886.
Li AH, Zhang JM, Xie YK. Human acupuncture points mapped in rats are associated with excitable muscle/skin-nerve complexes with enriched nerve endings. Brain Res. 2004;1012:154–159.
Langevin HM, Yandow JA. Relationship of acupuncture points and meridians to connective tissue planes. Anat Rec. 2002; 269:257–265.
Moffet HH. Sham acupuncture may be as efficacious as true acupuncture: a systematic review of clinical trials. J Altern Complement Med. 2009;15:213–216.
Scharf HP, Mansmann U, Streitberger K, Witte S, Krämer J, Maier C, et al. Acupuncture and knee osteoarthritis: a three-armed randomized trial. Ann Intern Med. 2006;145:12–20.
Lee A, Fan LT. Stimulation of the wrist acupuncture point P6 for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev. 2009;2:CD003281.
Koh RB, Isaza N, Xie H, Cooke K, Robertson S. Effects of maropitant, acepromazine, and electroacupuncture on vomiting associated with administration of morphine in dogs. J Am Vet Med Assoc. 2014;244:820–829.
Wen G, He X, Lu Y, Xia Y. Effect of acupuncture on neuro-transmitters/modulators. In: Xia Y, Cao X, Wu G, Cheng J, eds. Acupuncture Therapy for Neurological Diseases: A Neurobiological View. Beijing: Tsinghua University Press and Berlin: Springer; 2010:120–142.
Robinson NG. The need for consistency and comparability of transpositional acupuncture points across species. Am J Trad Chin Vet Med. 2006;1:14–21.
Langevin HM, Wayne PM, MacPherson H, Schnyer R, Milley RM, Napadow V. Paradoxes in acupuncture research: strategies for moving forward. Evid Based Complement Altern. 2011. https://doiorg.publicaciones.saludcastillayleon.es/10.1155/2011/180805, 11 pages [Date accessed: April 14th, 2017].
Birch S, Alraek T, Kim KH, Lee MS. Placebo-controlled trials in acupuncture: problems and solutions. In: Leung SW, Hu Hao, eds. Evidence-based Research Methods for Chinese Medicine. Singapore: Springer Science + Business Media; 2016.
Author information
Authors and Affiliations
Corresponding author
Additional information
Disclosure statement
The authors declare that they have no conflicts of interest and no financial interests related to the material in this manuscript.
Rights and permissions
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
About this article
Cite this article
Yang, T., Shmalberg, J., Hochman, L. et al. Comparison of Point Placement by Veterinary Professionals with Different Levels of Acupuncture Training in a Canine Cadaver Model. Innov. Acupunct. Med. 10, 360–370 (2017). https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.jams.2017.08.002
Received:
Accepted:
Published:
Issue Date:
DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1016/j.jams.2017.08.002