twitter

Saturday 26 November 2016

'Asking the Right Question'. A Comparison of Two Approaches to Gathering Data on 'Herbals' Use in Survey Based Studies

. 2016; 11(2): e0150140.
Published online 2016 Feb 25. doi:  10.1371/journal.pone.0150140
PMCID: PMC4767213


Pratibha V. Nerurkar, Editor
1The Division of Applied Health Sciences, The University of Aberdeen, Aberdeen, United Kingdom, AB25 2ZD
2Pharmacy Department, Women's Hospital-HMC, Doha, Qatar
3The Department of Obstetrics and Gynaecology, The University of Aberdeen, Aberdeen, United Kingdom, AB25 2ZD
4The Department of Obstetrics and Gynaecology, Ninewells Hospital and Medical School, NHS Tayside, Dundee, United Kingdom, DD1 9SY
5Hamad Medical Corporation, Post Box – 3050, Doha, Qatar
6School of Pharmacy and Life Sciences, Robert Gordon University, Aberdeen, United Kingdom, AB10 7GJ
College of Tropical Agriculture and Human Resources, University of Hawaii, UNITED STATES
Competing Interests: The authors have declared that no competing interests exist.
Conceived and designed the experiments: JSM DS ARP AS BP MAH. Performed the experiments: ARP AS BP JSM. Analyzed the data: ARP DS JSM AS BP MAH. Contributed reagents/materials/analysis tools: JSM DS ARP BP AS. Wrote the paper: JSM DS ARP AS BP MAH.

Abstract

Background

Over the last decade academic interest in the prevalence and nature of herbal medicines use by pregnant women has increased significantly. Such data are usually collected by means of an administered questionnaire survey, however a key methodological limitation using this approach is the need to clearly define the scope of ‘herbals’ to be investigated. The majority of published studies in this area neither define ‘herbals’ nor provide a detailed checklist naming specific ‘herbals’ and CAM modalities, which limits inter-study comparison, generalisability and the potential for meta-analyses. The aim of this study was to compare the self-reported use of herbs, herbal medicines and herbal products using two different approaches implemented in succession.

Methods

Cross-sectional questionnaire surveys of women attending for their mid-trimester scan or attending the postnatal unit following live birth at the Royal Aberdeen Maternity Hospital, North-East Scotland. The questionnaire utilised two approaches to collect data on ‘herbals’ use, a single closed yes/no answer to the question “have you used herbs, herbal medicines and herbal products in the last three months”; and a request to tick which of a list of 40 ‘herbals’ they had used in the same time period.

Results

A total of 889 responses were obtained of which 4.3% (38) answered ‘yes’ to herbal use via the closed question. However, using the checklist 39% (350) of respondents reported the use of one or more specific ‘herbals’ (p<0.0001). The 312 respondents who reported ‘no’ to ‘herbals’ use via the closed question but “yes” via the checklist consumed a total of 20 different ‘herbals’ (median 1, interquartile range 1–2, range 1–6).

Conclusions

This study demonstrates that the use of a single closed question asking about the use of ‘herbals’, as frequently reported in published studies, may not yield valid data resulting in a gross underestimation of actual use.

Introduction

The increasing use of herbs, herbal medicines and herbal products (‘herbals’) to maintain health and treat a variety of medical conditions has generated significant academic interest. Since 2005 there has been a 20 fold increase in the number of systematic reviews and a two fold increase in prevalence surveys relating to ‘herbals’ published in peer reviewed journals (See Fig 1).
Fig 1
Number of hits over the last 10 years for Medline title search for ‘systematic review’ and ‘herb*’ and number of hits over the last 10 years for Medline abstract search of (‘questionnaire*’ or ‘survey*’) ...
Given the paucity of robust data regarding the safety, efficacy and potential for interaction with prescribed medication, the widespread use of ‘herbals’ by the public should be of interest to the medical profession and regulatory authorities []. Of particular concern may be use by pregnant women, who may endanger both their own health and that of their baby via potential herbs-drug interactions or direct toxicity from active herbal ingredients or toxic adulterants [].
A key limitation of most published studies reporting the use of ‘herbals’ is the failure to clearly define the scope of ‘herbals’ investigated. A recent systematic review of prevalence studies reporting the use of ‘herbals’ and other complementary and alternative medicine (CAM) modalities by pregnant women, identified that appropriate definitions of ‘herbals’ and CAMs were only provided in twelve of twenty-two studies []. Furthermore, detailed checklists naming specific ‘herbals’ and CAM modalities were described in only ten studies, with little uniformity in content across studies []. These failings limit direct inter-study comparison, generalisability, data pooling and the potential for meta-analyses.
This situation is not helped by the complex and major differences in the terms and definitions of ‘herbals’ originating from different regulatory authorities [] (Table 1). In addition there are differences between countries in the regulatory classification of ‘herbals’. For instance in the United Kingdom some herbal products are classified as food supplements or cosmetics and others as medicines, while in the USA all herbal medicines and products are described as ‘Dietary Supplements’ [].
Table 1
Definitions of herbs, herbal medicines, herbal products and Dietary Supplements used by the World Health Organisation, Medicines and Healthcare products Regulatory Agency (MHRA, United Kingdom) and the Food and Drug Administration (FDA, United States ...
As the majority of studies assessing the use of ‘herbals’ use a questionnaire based approach, it is essential that research participants clearly understand and correctly interpret the questions asked. Therefore terms used in the questionnaire should be explicitly defined to ensure the collection of valid data.
In an attempt to standardise data collection and reporting in CAM related studies, Quandt et al described the development of a CAM questionnaire (I-CAM-Q) to assess self-reported CAM use [, ]. The I-CAM-Q has four sections covering: visits to health care providers; complementary treatments received from physicians; use of herbal medicine and dietary supplements; and self-help practices. However, there is little specific reference to ‘herbals’ other than: ‘have you visited a herbalist in the last 12 months?’; ‘have you received herbs from a physician in the last 12 months?’ and ‘list up to three herbs/herbal medicines you have used in the last 12 months’. Of note there is inconsistency in the use of the terms ‘herbs’ and ‘herbal medicines’, there is no definition of these terms and there is no list provided of specific ‘herbals’ from which respondents may select. These are major issues which may impact the internal validity of the data.
Therefore the aim of our research was to assess the potential for differences in self-reported use of ‘herbals’ using two different questioning approaches; single closed question and a list of specific ‘herbals’ administered consecutively.

Methods

Data assessing the use of CAM collected from women (332) attending for their mid-trimester (18–21 weeks) scan and women with a live birth admitted to the postnatal unit (557) at the Royal Aberdeen Maternity Hospital, North-East Scotland were combined. Data collection was completed in 2012 and study methods have been reported in detail elsewhere [, ]; brief study details are given for completeness.
Questionnaires were based on the findings of our systematic review assessing the quality of study methodologies used to derive data reporting CAM use during pregnancy ([], S1 Questionnaire]. The questionnaire was written in English only and tested for face and content validity by a panel of researchers, healthcare professionals, pregnant and postpartum women prior to piloting. Amongst other items, the questionnaire utilised two different approaches to collect data on ‘herbals’ used.
The first of these was one closed question, which asked the participant to tick all of a possible 23 different Complementary and Alternative Therapies which they had used (S1 Table of Complementary and Alternative Therapies). In the antenatal and postnatal studies participants were asked to: “Please tell us if you have used any of the following Complementary and Alternative Therapies during the last three months?” The question was followed by the request to “Please tick all Complementary and Alternative Therapies that you have used. If you haven’t heard of some of the names before, don’t worry. For each of the Complementary and Alternative Therapies you have used, please tell us why you used it and how you heard about it (doctor, pharmacists, midwife, family friend, internet, magazine)”. At a later stage in the questionnaire, participants were presented with an extensive list (sourced from the Medicines and Healthcare products Regulatory Agency (MHRA, UK)) of 40 ‘herbals’ [] (S2 Table of Herbal and Natural Products) and asked in the antenatal and postnatal studies to “Please tell us if you have used any of the following Herbal and Natural Products during the last three months?; The question was then followed by the request to “Please tick all Herbal and Natural Products that you have used. If you haven’t heard of some of the names before, don’t worry. For each of the Herbal and Natural Products you have used, please tell us why you used it and how you heard about it (doctor, pharmacists, midwife, family friend, internet, magazine).
Data were analysed using descriptive statistics and a two tailed Pearson’s Chi square test to determine associations between the two different question approaches and the proportion answering yes to ‘herbals’ use. A P value <0.05 was considered statistically significant.

Ethics Statement

This research was approved by National Health Service North of Scotland Research Ethics Committee and National Health Service Grampian Research and Development Committee on June 27, 2011 (REC 11/ AL/0094). As the Ethics Committee required the survey questionnaires to be fully anonymous and returned directly by respondents in post-paid envelopes with no record of identifiable data, oral rather than written consent to participate was deemed appropriate.

Results

Eight hundred and eighty-nine respondents completed the questionnaire giving an overall response rate was 71%. Respondent demographics are reported in Table 2. Of 889 respondents, only 4.3% (38) reported use of ‘herbals’ via the closed question. However, using the detailed list 39% (350) of respondents reported the use of one or more specific ‘herbals’. Therefore 312 (35%) respondents who reported ‘no’ to ‘herbals’ use via the closed question actually reported “yes” to the use of herbs, herbal medicines or herbal products via the detailed list. This difference is statistically significant, p<0.0001.
Table 2
Study population demographics (n = 889).
The 312 respondents who initially reported “no” actually consumed or used a total of 20 different ‘herbals’ (median 1, interquartile range 1–2, range 1–6) (See Fig 2).
Fig 2
Bar chart of the actual number of different herbs, herbal medicines and herbal products taken or used by those responding ‘no’ to the closed question “have you used herbs, herbal medicines or herbal products in the last three months” ...
The most frequently reported ‘herbals’ used were: raspberry tea or capsules 61% (126); ginger 29% (89); cranberry 22% (70); chamomile 16% (49); peppermint 12% (36); eucalyptus 8.3% (26); aloe 6.7% (21); grapefruit 6.4% (20); senna 5.4% (17); echinacea 4.5% (14); garlic beyond cooking 3.5% (11); ginseng 1.3% (4); 0.6% (2) each for aconite, nettle root, dong quai; 0.3% (1) each for barberry, bee pollen, blue cohosh, ginkgo biloba, and kava.
Binary logistic regression did not identify and significant differences, in terms of demographics, between the two study patient cohorts.

Discussion

Our key finding is that using the closed question approach, as recommended in the I-CAM-Q [, ], grossly underestimated the true use of ‘herbals’ by the public. However employing a detailed list of ‘herbals’ generated a significant tenfold increase in the number of individuals reporting actual use.
To our knowledge this is the first study to compare two different approaches administered consecutively to the same cohort to determine the self-reported use of ‘herbals’.
In light of these findings there are key implications for the interpretation and generalizability for much of the research published in this area. It should not be surprising that data reported for the use of CAM modalities and ‘herbals’ during pregnancy are highly variable given that less than half of published prevalence studies described detailed checklists []. Of note, the I-CAM-Q asks respondents to name up to three herbs/herbal medicines taken or used in the past 12 months, therefore assuming that respondents can interpret the terms herbs and herbal medicines, and are able to categorise accordingly. Our data provide robust evidence that such an approach is likely to yield a gross and highly significant underestimate of true prevalence, threatening the internal validity of the data.
Population surveys such as the I-CAM-Q are routinely used to gain a picture of public practice or belief; however it is clear from the results of this study that two simple questions, which might at first glance be expected to give similar results, gave rise to significantly different responses. Although writing survey questions may initially appear simple the current literature would suggest otherwise []. The question setter must not only have a clear idea of the question intent and the likely responses, but also ensure that questions are written using language which the respondent can understand and process as intended []. Failure to follow the basic principles for questionnaire design and item writing described in the literature will inevitably give rise to misleading or erroneous survey results.
To ensure accurate and robust reporting of ‘herbals’ use during pregnancy the use of a detailed list is clearly necessary, however this may be problematic given that the number of available ‘herbals’ is increasing []. The development and application of specific checklists will require regular review and updating to acknowledge regional and temporal variability [, ] in ‘herbals’ use.
Although many ‘herbals’ have been used for centuries, underestimating the level of use during pregnancy is of clinical importance given that several of these agents have been directly associated with or have the potential to cause both maternal and fetal harm [, ].

Strengths and Limitations

A key strength of this study is the consecutive administration of two different approaches to identify ‘herbals’ use in the same population.
A possible limitation however is the use of a study population in the North of Scotland, which may limit the generalizability of the findings. However our study population was relatively diverse in terms of ethnicity, parity and health status hence there is no reason to suspect that the outcome would be different in other populations. It is also possible that poor health literacy may have contributed to the significant differences we observed, however both approaches were administered concurrently to the study cohort and our population was relatively well educated, with over three quarters reporting a college or university education. Therefore it is unlikely that poor health literacy was responsible for the differences we observed between the approaches used.

Conclusion

This study has demonstrated the need to ensure that detailed checklists of ‘herbals’ are used in all prevalence studies.
Our findings may also have relevance for the practicing clinician who should adopt detailed checklists when asking about a patient’s use of ‘herbals’.

Supporting Information

S1 Questionnaire

Exemplar Antenatal CAM in Pregnancy Questionnaire.
(DOC)

S1 Table

Table of Complementary and Alternative Therapies.
Complementary and Alternative Therapies listed for the Question “Please tell us if you have used any of the following Complementary and Alternative Therapies?”
(DOCX)

S2 Table

Table of Herbal Medicines and Products.
Herbal Medicines and Products listed for the Question “Please tell us if you have used any of the following Herbal and Natural Products?
(DOCX)

Funding Statement

This work was funded from internal institutional resource. Prof James McLay is employed by the University of Aberdeen, Prof Derek Stewart is employed by the Robert Gordon University, Drs A.R. Pallivalappila and M. AL Hail are employed by the Hamad Medical Corporation and Drs A. Shetty and B. Pande are employed by NHS Scotland. None of the institutional funders had a role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data Availability

All relevant data are within the paper and its Supporting Information files.

References

1. Ernst E. Prevalence of use of complementary/alternative medicine: a systematic review. Bull World Health Organ 2000; 78 (2):252–257. [PMC free article] [PubMed]
2. Harris PE, Cooper KL, Relton C, Thomas KJ. Prevalence of complementary and alternative medicine (CAM) use by the general population: a systematic review and update. Int J Clin Pract 2012. October; 66(10):924–939. doi: 10.1111/j.1742-1241.2012.02945.x [PubMed]
3. Frass M, Strassl RP, Friehs H, Mullner M, Kundi M, Kaye AD. Use and acceptance of complementary and alternative medicine among the general population and medical personnel: a systematic review. Ochsner J 2012. Spring;12(1):45–56. [PMC free article] [PubMed]
4. Fischer FH, Lewith G, Witt CM, Linde K, von Ammon K, Cardini F, et al. High prevalence but limited evidence in complementary and alternative medicine: guidelines for future research. BMC complementary and alternative medicine. 2014; 14(1), 46. [PMC free article] [PubMed]
5. Fugh-Berman A, Kronenberg F. Complementary and alternative medicine (cam) in reproductive-age women: a review of randomized controlled trials. Reprod Toxicol. 2003; 17(2): 137–152. [PubMed]
6. Marcus DM, Snodgrass Wr. Do no harm: avoidance of herbal medicines during pregnancy. Obstet Gynecol. 2005; 5(1): 1119–1122. [PubMed]
7. Ernst E. Herbal medicinal products during pregnancy: are they safe? BJOG.2002; 109:227–235; [PubMed]
8. Chuang CH, Doyle P, Wang JD, Chang PJ, Lai JN, Chen PC. Herbal Medicines Used During the First Trimester and Major Congenital Malformations An Analysis of Data from a Pregnancy Cohort Study. Drug Safety 2006; 29 (6): 537–548; [PubMed]
9. Memory Elvin-Lewis Should we be concerned about herbal remedies. J Ethnopharmacology. 2001; 75:141–164. [PubMed]
10. Dugoua JJ, Mills E, Perri D, Koren G. Safety and Efficacy of Ginkgo (Ginkgo Biloba) during Pregnancy and Lactation. Can J Clin Pharmacol 2006; 13: 277–84. [PubMed]
11. Ko R J. Adulterants in Asian Patent Medicines. N Eng J Med 1998; 399: 847. [PubMed]
12. Briggs GG., Freeman Roger K., Yaffe JS. Drugs in pregnancy and lactation: a reference guide to fetal and neonatal risk. Lippincott Williams & Wilkins, 2012.
13. Moussally K, Berard A. Exposure to herbal products during pregnancy and the risk of preterm birth. Eur J Obstetrics & Gynecology and Reproductive Biology. 2010; 150: 102–108. [PubMed]
14. Cuzzolin L, Francini-Pesenti F, Verlato G, Joppi M, Baldelli P, Benoni G. Use of herbal products among 392 Italian pregnant women: focus on pregnancy outcome. Pharmacoepidemiology and Drug Safety 2010; 19: 1151–1158 doi: 10.1002/pds.2040 [PubMed]
15. Facchinetti F, Pedrielli G, Benoni G, Joppi M, Verlato G, Dante G, et al. Herbal supplements in pregnancy: unexpected results from a multicentre study. Human Reproduction 2012; 27, 11: 3161–3167. doi: 10.1093/humrep/des303 [PubMed]
16. Boivin J, Schmidt L. Use of complementary and alternative medicines associated with a 30% lower on going pregnancy/live birth rate during 12 months of fertility treatment. Human Reproduction 2009; 24, 7: 1626–1631. doi: 10.1093/humrep/dep077 [PubMed]
17. Pallivalappila AR, Stewart D, Shetty A, Pande B, McLay JS. Complementary and Alternative Medicines Use during Pregnancy: A Systematic Review of Pregnant Women and Healthcare Professional Views and Experiences. Evidence-Based Complementary and Alternative Medicine. 2013; Article ID 205639. doi: 10.1155/2013/205639 [PMC free article] [PubMed]
18. General Guidelines for Methodologies on Research and Evaluation of Traditional Medicine. World Health Organisation; Geneva: http://www.who.int/medicines/areas/traditional/definitions/en/ Accessed 04/2015
19. The Human Medicines Regulations 2012 No 1916 (14 August 2012). http://www.legislation.gov.uk/uksi/2012/1916/regulation/217/made Accessed 02/2016
20. Dietary Supplement Health and Education Act of 1994 Public Law 103–417 103rd Congress. http://ods.od.nih.gov/About/DSHEA_Wording.aspx#sec3 Accessed 02/2016
21. The Medicines and Healthcare products Regulatory Agency (MHRA). Herbal medicines regulation. 2013; Available at: http://www.mhra.gov.uk/Howweregulate/Medicines/Herbalmedicinesregulation/ Accessed 02/2016
22. Barber S. Regulation of herbal medicines. 2014; Available at: http://www.parliament.uk/briefing-papers/SN06002/regulation-of-herbal-medicines Accessed 02/2016
23. Dunn JD, Cannon HE, Lewis T, Shane-McWhorter L. Development of a complementary and alternative medicine (CAM) pharmacy and therapeutics (P&T) subcommittee and CAM guide for providers. J Manag Care Pharm 2005;11 (3):252–258. [PubMed]
24. Quandt SA, Verhoef MJ, Arcury TA, Lewith GT, Steinsbekk A, Kristoffersen AE, et al. Development of an international questionnaire to measure use of complementary and alternative medicine (I-CAM-Q). J Alternative and Complementary Medicine. 2009; 15, no. 4: 331–339. [PMC free article] [PubMed]
25. Re ML, Schmidt S, Güthlin C. Translation and adaptation of an international questionnaire to measure usage of complementary and alternative medicine (I-CAM-G). BMC complementary and alternative medicine. 2012; 12(1): 259. [PMC free article] [PubMed]
26. Pallivalappila AR, Stewart D, Shetty A, Pande B, Singh R, Mclay JS. Complementary and Alternative Medicine Use During Early Pregnancy. Eur J Obstet Gynecol RB. 2014. doi: 10.1016/j.ejogrb.2014.08.017 [PubMed]
27. Pallivalapila AR, Stewart D, Shetty A, Pande B, Singh R, McLay JS. Use of complementary and alternative medicines during the third trimester. Obstetrics & Gynecology 2015; 125 (1):204–211. [PubMed]
28. Guidance: Herbal medicines granted a traditional herbal registration. Published 31 December 2014. https://www.gov.uk/government/publications/herbal-medicines-granted-a-traditional-herbal-registration-thr/herbal-medicines-granted-a-traditional-herbal-registration Accessed 02/2016
29. Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons; 2014. August 6.
30. Rossi PH, Wright JD, Anderson AB. Handbook of Survey Research. Academic Press; October 22, 2013
31. Nardi PM. Doing survey research. Routledge; 2015. November 17
32. Eardley S, Bishop F, Prescott P, Cardini F, Brinkhaus B, Santos-Rey K, et al. A systematic literature review of complementary and alternative medicine prevalence in EU. Forschende Komplementmed 2012;19 (suppl 2):18–28. [PubMed]
33. Rossi E, Picchi M, Panozzo M, Di Stefano M, Baccetti S. Integration of homeopathy and complementary medicine in the public health system in Italy: national regulation and regional experiences. J Med Pers. 2015; 13: 45–54.
34. Chuang CH, Doyle P, Wang JD, Chang PJ, Lai JN, Chen PC. Herbal Medicines Used During the First Trimester and Major Congenital Malformations An Analysis of Data from a Pregnancy Cohort Study. Drug Safety 2006; 29 (6): 537–548; [PubMed]
35. Memory Elvin-Lewis Should we be concerned about herbal remedies. J Ethnopharmacology. 2001; 75:141–164. [PubMed]
36. Holst L, Haavik S, Nordeng H. Raspberry leaf–Should it be recommended to pregnant women? Complementary therapies in clinical practice. 2009; 15(4), 204–208. doi: 10.1016/j.ctcp.2009.05.003 [PubMed]
37. Langhammer AJ, Nilsen OG. In vitro Inhibition of Human CYP1A2, CYP2D6, and CYP3A4 by Six Herbs Commonly Used in Pregnancy. Phytotherapy Research. 2014; 28(4), 603–610. doi: 10.1002/ptr.5037 [PubMed]
38. Smeriglio A., Tomaino A., & Trombetta D. (2014). Herbal Products in Pregnancy: Experimental Studies and Clinical Reports. Phytotherapy Research.2014; 28, 8:1107–1116. doi: 10.1002/ptr.5106 [PubMed]
39. Johnson JR, Makajo E, Ho S, Xiong B, Crankshaw DJ, Holloway AC. Effect of maternal raspberry leaf consumption in rats on pregnancy outcome and the fertility of the female offspring. Reprod Sci; 2009; doi: 10.1177/1933719109332823) [PubMed]