Types of Control in Acupuncture Clinical Trials Might Affect the Conclusion of the Trials: A Review of Acupuncture on Pain Management

Abstract

Analgesic effects of acupuncture have been extensively studied in various clinical trials. However, the conclusion remains controversial, even among large scale randomized controlled trials. This study aimed to evaluate the association between the conclusion of the trials and the types of control used in those trials via systematic review. Published randomized controlled trials of acupuncture for pain were retrieved from electronic databases (Medline, AMED, Cochrane libraries, EMBASE, PsycINFO, Clinicaltrials.gov, and CAB Abstracts) using a prespecified search strategy. One hundred and thirty-nine studies leading to 166 pairs of acupuncture-control treatment effect comparisons (26 studies comprised of 53 intervention-control pairs) were analyzed based on the proportion of positive conclusions in different control designs. We found that treatment effects of acupuncture compared with nontreatment controls had the highest tendency to yield a positive conclusion (84.3%), compared with nonneedle-insertion controls (53.3%).

Whereas with needle-insertion controls, the lowest tendency of positive conclusions was observed (37.8%). Consistently, in studies reporting successful blinding, a higher tendency of positive findings on the treatment effect of acupuncture was found in the noninsertion sham controls compared with that in the insertion sham controls. We conclude that the type of control is likely to affect the conclusion in acupuncture analgesic trials. Appropriate control should be chosen according to the aims of studies.

1. Introduction

The number of randomized controlled trials (RCTs) conducted on acupuncture have dramatically increased over the past decade. The efficacy of acupuncture for pain is one of the most interesting outcomes in studies. Although many basic science studies have revealed the analgesia mechanisms of acupuncture [1,2], the efficacy of acupuncture remains controversial in clinical trials, e.g., in knee osteoarthritis (KOA) [3–8]. The diverse mechanisms and complicated manual procedures involved in acupuncture treatment have contributed to the challenges of evaluating acupuncture trials [9]. For example, acupuncture produces a specific physiological effect and nonspecific needling effect (e.g., diffuse noxious inhibitory control) during the treatment [10]. Patient expectations, acupuncturist experience, number and specificity of acupoints, depth of needling, and dosage of acupuncture (duration, frequency, and time) also affect the efficacy of acupuncture analgesia in RCTs [11]. The benefits during the treatment are usually explained by: (1) treatment effects; (2) nonspecific effects; or (3) spontaneous remissions [12,13]. A proper control or controls, e.g., waitlist, non-insertion sham acupuncture, and insertion sham acupuncture, are utilized to evaluate the true effects in RCTs [9].

Arguments have been raised on the efficacy of acupuncture controls [14–16]. Meng et al [17] reviewed acupuncture RCTs on pain published in 2006–2007 and found that trials using noninsertion shams yielded more positive outcomes (6 of 7 trials) than those using insertion shams (2 of 8 trials). Madsen et al [18] found that the type of placebo acupuncture was not associated with the estimated analgesic effect of acupuncture. In this study, we aimed to examine whether positive conclusion is correlated with the type of controls in RCTs of acupuncture for pain. We systematically reviewed clinical trials of acupuncture for pain from 2004 to 2014. The association between the type of controls used in these studies and conclusion of acupuncture efficacy were further analyzed.

2. Materials and methods

2.1. Database

A systematic search of RCTs with acupuncture was conducted to evaluate the proportion of positive conclusions in the different controls in RCTs. The search strategy was defined as below. Databases searched included Medline, AMED, Cochrane libraries, EMBASE, PsycINFO, Clinicaltrials.gov, and CAB Abstracts.

2.2. Search strategy

The search keywords were as follows: “acupuncture*”, “acupoint*”, “acupress*”, “meridian*”, “needle*”, “sham acupuncture”, “placebo acupuncture”, “control acupuncture”, “acupuncture control”, and “pain”. Studies were limited to RCTs and journals in Science Citation Index (SCI). The search was conducted in March 2015.

2.3. Screening

The retrieved studies were imported into Endnote and any duplicates were removed. The abstracts of the studies were screened, followed by full-text screening according to the selection criteria below. The screening was performed by two individuals. Discrepancies were resolved by discussion with a third reviewer. Information on the type of controls and acupuncture efficacy conclusion from eligible studies were extracted according to the definition of outcomes.

2.4. Selection criteria

2.4.1. Inclusion criteria

Studies: (1) were RCTs; (2) used pain score as an outcome; (3) used needling acupuncture (traditional acupuncture, electro-acupuncture, and medical acupuncture) as the major intervention (not restricted to auricular acupuncture and scalp acupuncture as the secondary intervention); and

(4) were published from 2004 to 2014.

2.4.2. Exclusion criteria

Studies: (1) used bee venom acupuncture as the intervention; (2) used acupoint injection as the intervention; (3) of poor quality design (unclear randomization method, incorrect concealment, and individual assessment), with low risk items less than five of seven (according to risk bias assessment tool in Cochrane review handbook); and (4) used active treatment of any acupuncture modalities (e.g., active acupuncture, auricular acupuncture, etc.) as control(s).

2.5. Outcomes

2.5.1. Type of acupuncture controls

We classified acupuncture controls into several types according to the purpose of controls: (1) “nontreatment” control: patients usually received nontreatment, delayed treatment (waiting list), usual care, or/and rescue medication in consideration of medical ethics; (2) noninsertion sham: these do not penetrate the skin, but usually use the blunt end of the acupuncture needles, noninsertion sham devices (e.g. Streitberger or Park sham devices), and other needle-resembling devices such as toothpicks and needling guiding tubes; (3) insertion sham: usually involves a superficial insertion of needles to acupoints or nonacupoints; (4) combined noninsertion and insertion sham; and (5) comparator (positive control): refers to active treatments, such as specific mediations and physiotherapies, some usual care, or standardized care, etc., which were thought to be effective.

Usual care refers to standardized patient care practices that have not been validated by rigorous clinical evidence, or uniform practices that have not been identified as the “best current therapy” in clinical practices while the individualized cares are available [19]. The role of usual care remains controversial [19,20]. In pain trials, usual care commonly refers to standardized management that is not sufficient to kill pain but is routinely provided to patients [19,20]. In this study, if both arms of intervention and control used usual care, we classified the type of control into no treatment control. Also participants in no treatment or waitlist control group have access to nonstudy healthcare services [21]. There are other study designs where usual care serves as the comparator, and is only used in the control arm but not the intervention arm. In that case, we classified usual care as a positive control or comparator.

If a study contained two or more controls, information on each acupuncture-control comparison pair was extracted according to the control types.

2.5.2. Type of conclusions in clinical trials

Positive conclusion was defined as acupuncture showing statistically significant superiority to the control (p < 0.05) in the primary outcome of clinical studies. If no primary outcome was stated in the studies, the general conclusion of the study was judged as a positive conclusion when it indicated acupuncture was better than the control.

Negative conclusion was defined as acupuncture not showing statistically significant superiority to the control (p ≥ 0.05) in the primary outcome of clinical studies. If no primary outcome was stated in the studies, the general conclusion of the article was judged as a negative conclusion if it indicated acupuncture was not better than the control.

An inconclusive conclusion was defined as acupuncture showing statistically significant superiority to the control in some primary outcomes but not in all primary outcomes. If no primary outcome was stated in the studies, the general conclusions of the study was as inconclusive when it indicated acupuncture was somewhat better than the control but not in all outcomes.

3. Results

According to the search strategy, 2,934 studies were retrieved. The flowchart of screening is shown in Fig. 1. One hundred and thirty-nine studies were included with 166 pairs of intervention controls as 26 studies contributed 53 intervention-control pairs. The following analysis was performed according to 166 intervention-control pairs in 139 studies. Using Fisher’s exact test, there was s statistically significant relationship between the type of control and study conclusion (p < 0.0001; Table 1). Robustness of the result was demonstrated by sensitivity analysis that excluded the combined control studies and/or inconclusive studies.

Table 1 Types of control by study conclusion in acupuncture clinical trials.

Figure 1

Flowchart of screening. One hundred and thirty-nine studies with 166 pairs of intervention-controls were analyzed. RCT = randomized controlled trial; SCI = Science Citation Index.

3.1. Nontreatment control

Patients in this type of control usually received nontreatment or delayed treatment (called waiting list). Usual care or rescue medications were introduced in both the treatment group and nontreatment control group during the clinical studies. As shown in Table 1, 84.3% of intervention nontreatment pairs in clinical trials had positive efficacy conclusions (43/51). A negative conclusion was yielded in 11.8% of them (6/51). Two pairs of intervention nontreatment were inconclusive.

3.2. Noninsertion sham control

The noninsertion control resembles the real acupuncture needling procedure but does not really penetrate the skin. Many types of noninsertion control have been used in acupuncture trials, e.g., empty guiding tube, semiblunt needling, toothstick, nonpenetrating needle devices, etc. [6,22–25]. As shown in Table 1, 53.3% of intervention noninsertion sham pairs in clinical trials had positive efficacy conclusions (16/30), while 43.3% of them yielded negative conclusions (13/30). One pair of intervention noninsertion shams were inconclusive (3.3%).

3.3. Insertion sham acupuncture control

The needle-insertion sham acupuncture control usually penetrates the skin but at nonacupoints or the acupoints which are believed to have no specific effect [4,7,26–28]. As shown in Table 1, 37.8% of intervention-insertion sham pairs in clinical trials had positive efficacy conclusions (14/ 37), while 54.1% of them yielded negative conclusions (20/ 37). Three pairs of intervention-insertion shams were inconclusive (8.1%).

3.4. Combined controls

As shown in Table 1, two studies used the combined controls. Berman et al [3] used noninvasive guide tubes at local acupoints around the knee and lower leg and inserted two needles on the abdomen at points away from meridians in a clinical trial of KOA. Another study used double-dummy design to evaluate the efficacy of acupuncture for migraine prophylaxis [29]. The treatment group consisted of real acupuncture and placebo medication, and the control group had true medication and sham acupuncture (perpendicularly needling at sham acupoints with lifting, thrusting, and twirling to obtain DeQi) [29]. Both of them had positive conclusions of acupuncture efficacy.

3.5. Positive comparison

Medications, physiotherapies, and other treatments were used as comparators in many studies. As shown in Table 1, 56.5% of intervention-insertion sham pairs in clinical trials had positive efficacy conclusions (26/46), while 34.8% of them yielded negative conclusions (16/46). Four pairs of intervention-comparison sham were inconclusive (8.7%).

3.6. Positive conclusion in blinding validated studies

Only 12 studies reported blinding validation tests in the clinical trials, accounting for 7.2% of all included studies. All studies reported successful blinding. Studies that used insertion sham controls had 100% negative conclusions. Among studies that used noninsertion sham controls, 28.6% had positive conclusions and 57.1% had negative conclusions (Table 2). However, the relationship between study control type and study conclusion in these studies was not significant (Fisher’s exact test, p = 0.47).

Table 2 Conclusions of studies with blinding credibility tested.

4. Discussion

In this study, we systematically reviewed RCTS that studied the efficacy of acupuncture for pain. Potential association between the conclusions of acupuncture efficacy and the types of controls was analyzed. We found that studies had the highest tendency to yield positive conclusions (84.3%) when nontreatment controls were used, compared with a lower tendency (53.3%) observed in the noninsertion controls, and lowest tendency (37.8%) in the insertion controls. Consistently, in studies reporting successful blinding, a higher tendency of positive conclusion was found in non-insertion sham controls compared with that in insertion sham controls.

In clinical practice, acupuncture analgesia may be explained by various effects, such as the specific therapeutic effect, nonspecific physiology effect, placebo effect, or disease spontaneous remission. These effects are commonly distinguished by adopting specific controls or are excluded by appropriate trial design step by step.

The nontreatment control determines whether the disease has spontaneous remission. It had the highest positive conclusion of acupuncture efficacy and the cost is lower than RCTs using other controls such as sham control. It is more feasible to conduct a clinical trial using nontreatment control compared with using other types of controls. With this advantage, nontreatment control is recommended to establish the adequate dose of acupuncture (e.g., number of acupoints, frequency, and duration of acupuncture), optimize the duration of treatment, select proper measurements and measurement time points, or examine the safety in a pilot study or at the early stage of developing a certain acupuncture treatment.

However, patients assigned to receive nontreatment usually prefer to get real treatment. Their feeling worse in the disease condition for not having the opportunity to receive the real treatment is called nocebo effect [30]. The nocebo effect is regarded as negative placebo effect which has been raised from expectation and psychological conditioning [30]. Wait list control offers patients the same treatment as the treatment group after the patient completes treatment so that nocebo effect is minimized as much as possible. In fact, few studies restrict patients to take medications or other therapies if patients really need treatments. Taking into consideration the ethical issue and nocebo effect, usual care, medical education, or rescue medications are used as the “nontreatment” control [31,32].

Studies using the noninsertion controls have a higher tendency of positive conclusion compared with those using needle insertion controls in acupuncture for pain studies. It could be explained that needle insertion controls may produce more nonspecific physiological effects, e.g., the diffuse noxious inhibitory controls [10]. The difference in pain scale between acupuncture treatment groups and needle insertion controls is likely to be smaller than studies using noninsertion controls. However, noninsertion controls may reduce the success of blinding as patients with acupuncture experience are more likely to identify the sham treatment, which lowers patient expectancy and attendance. The noninsertion sham controls can be used for the short-term trials, e.g., acute pain study, or trials recruiting acupuncture naïve patients. Insertion sham controls are more similar to real acupuncture. In the reviewed studies, most of the studies used superficial needling, and needle points were selected out of the meridians, distal acupoints, or acupoints with no effects [4,7,26–28,33,34]. Needle manipulation should not be applied to partly reduce the nonspecific effect of insertion sham. However, this superficial, distal needling may also produce similar effects to real acupuncture. For example, Vas et al [35] used both needle insertion sham to study point specificity and noninsertion to control acupuncture technique. They found that all three treatmentsdreal acupuncture, insertion sham, and noninsertion shamdhad better effects than conventional treatment, and there was no significant difference among the three treatments [35].

To achieve the advantages of both insertion and non-insertion sham controls, Berman et al [3] applied a combined control in a KOA trial. The acupuncture treatment consisted of real needling at five local points, four distal points, and tapping plastic guiding tube at two sham points (noninsertion sham control) at the abdomen, and the sham control consisted of inserting two needles at sham points (insertion sham control) and tapping at nine real points (noninsertion placebo control) [3,13].

The masking effectiveness or the blinding credibility should be measured for both real acupuncture and sham acupuncture treatments. Only 7.2% of studies assessed blinding success. No study with blinding credibility assessed indicated unsuccessful blinding. In the KOA study, the combined control produced acceptable masking effects [3], 25% and 33% of the patients were unsure of their assignment in the real acupuncture or sham acupuncture group, and 67% and 58% believed that they were receiving true acupuncture (p = 0.06), respectively. In addition to the combined control, to avoid the nonspecific effect of needling, the number of needling should be minimized.

In some studies, treatments with positive effects, such as conventional medications or other active treatments (physiotherapies, radiotherapies, and chemotherapies, etc.) were introduced as the comparators, rather than controls, for acupuncture treatment. These comparators serve as “positive controls” so that the effectiveness of acupuncture can be measured. The proportion of positive conclusions in such studies was 56.5%. It could be varied with the strength of therapeutic effects of the comparator. If researchers choose strong positive comparators for acupuncture treatment, there would be less positive conclusions in the study. A double dummy design for acupuncture and comparator could enhance the blinding effect in clinical trials, e.g., introduce placebo medication in acupuncture and sham acupuncture in comparison groups [29].

There are limitations in this study. Firstly, we only studied the association between the control type and study outcome. Although we had excluded the potential influence from the methodological quality, a few factors might affect the study outcome, e.g., the dose of acupuncture intervention, the severity of disease, the experience of acupuncturists, the effectiveness of controls, the success of blinding, etc. The potential effects should be fully considered in the clinical trial design. Secondly, as pain is a very common symptom, it manifests in various diseases. The search strategy we used in the study might not have retrieved all acupuncture clinical trials which were related to pain management. In the retrieved studies, pain was the major complaint. The findings from these studies should mainly reflect the trend of association in control type and study outcome. Lastly, given the difficulties to obtain the full text of many non-SCI publications, we limited the search in SCI publications. The restriction of studies in SCI publications may lead to bias.

Selection of controls in acupuncture trials is likely to affect the study conclusion. Studies using nontreatment controls have the highest tendency of positive conclusions, followed by noninsertion controls, and the lowest tendency in insertion sham controls. To improve the quality of acupuncture trials, the control needs to be appropriately selected.