Critical Appraisal of ‘A mixed methods study to compare models of spirometry delivery in primary care for patients at risk of COPD’
Walters et al (2007) argue that spirometry, or the process of measuring lung function on the basis and/or speed of air that can be inhaled or exhaled, is important in the diagnosis of chronic obstructive pulmonary diseases (COPD). The authors conducted a six-month long research for individuals who are at risk of COPD through comparing two models of spirometry delivery. The research aimed at evaluating and comparing the effectiveness of trained nurse (TN) spirometry and usual care (UC) spirometry models of delivery. Mixed methods were used as tied to a comparative design of research. In this report, a critical appraisal of the original research will be discussed based on the evaluation tool for mixed method studies developed by Health Care Practice Research & Development Unit (HCPRDU). It is aimed that this report would provide a detailed appraisal on the basis of implications of the study, randomisation, group comparability, clinical validity of outcomes and generalisability.
Initially, there are a number of features of the study that is worth pointing out before conducting a detailed critical appraisal of the mixed method study. First, the figure illustrations of the study flow chart including the actual number of patients who are diagnosed using spirometry and the number of patients who refused spirometry makes it easier to follow the pathway of patients. Second, the figure illustration of the impact of spirometry on the diagnosis of COPD reflects a detailed comparison on the performance of the two models used. Third, the study reports baseline data in tabular form, making it easier to make comparisons between the subjects the two models used. Table 1, in fact, presents data at two endpoints. Fourth, the study also presents data of the patients’ reasons in accepting and declining spirometry in a tabular form for better understanding from the appraiser’s point of view. Fourth, the qualitative accounts of the general practitioners (GPs) regarding the spirometry delivery were also tabulated in two endpoints. Finally, the researchers accomplished approval of a study protocol from the Southern Health and Medical Human Research Ethics Committee and written consents are obtained from participating GPs.
The structured abstract helps in identifying the population wherein based on the eligible target population 531/904 patients underwent spirometry in the TN model and 87/1130 patients in the UC model. These are the target groups which represent at risk of COPD. Directly, one could realise that this may exclude a large number of otherwise eligible patients who could benefit from a spirometry intervention. Although the abstract hinted that the study was done in Australia, nevertheless, the problem is that it did not provide where in Australia in particular the study was conducted. Hence, it is likely that the incidence of COPD problems might be lower in this population than in any part of Australia where smoking rate is higher considering this age group. The aged group was identified as over 35 years. Notably, however, the identified potential difference does not result in the lack of clarity when it comes to the definition of the population. One should hold onto such point offered herein until later when I look at the applicability of the results to local population.
With the objective to increase recognition of airflow obstruction in primary care, the intervention is described as visiting trained nurse as compared with optimised usual care. Importantly, the terms visiting nurse and usual care contain the potential for significant variations when it comes to the quality and extent of the roles they played in the various care setting not just in the primary care setting. Although one may think that the details are already presented in the proposal, what the authors could have done is to position the experimental and control interventions (Caracelli and Greene, 1997) by means of outlining the functions of visiting nurses and usual carers in relation to the study. Moreover, there are significant differences between the two interventions. In the TN spirometry model, practice staff resort in invitation for opportunistic testing and advertising at the request of GPs and addressing cost issues whereas in the UC spirometry model, spirometry testing was done in accordance to usual practice protocol and was discretely made public, which could have contributed to differential effects. Both model undergone basic training of spirometry under the guidance of a physiologist and respiratory specialist physician.
Further, it would be also necessary to note that the study deals with spirometry which has inherent limitations. Spirometry largely depends on patient cooperation and should be repeated at least three times (Miller et al, 2005). The spirometry test was consisted of three expiratory maneuvers which meets the criteria of the American Thoracic Society (ATS) requirements of acceptability and repeatability. However, it did not mention how many times spirometry test was conducted, jeopardising reproducibility in the process. While it is stipulated in the study that the spirometer used has an advantage of stability, inbuilt quality assurance features and grade classifications, one has to employ a pragmatic approach on the extent of expertise of the nurses and usual carers in utilising the spirometer. The training provided is inadequate and is evident by the incompetence of GPs to interpret the findings. A “semi-black box” approach was evident wherein the interventions provide the input and the investigators provide for the output (Sydenstricker-Neto, 1997).
When it comes to the outcomes, one has to place amount of responsibility on the authors as they establish the principal outcomes of the study and over what period of time they are to be measured (Greene and Caracelli 1997). Results of study include 9 out of 531 participants and 2 out of 87 participants in TN and UC model, respectively had a new doctor recorded COPD diagnosis which leads the author to conclude that there is a substantial increase in and improved spirometry performance in the opportunistic testing compared to the usual care. The question now is: Are these particular outcomes valid and meaningful? Important to note is the fact that opportunistic testing was performed on 531 participants in the TN model and 87 participants were tested in the UC model. It is in this sense that one can argue that there will be a fundamental difference when it comes to the outcomes considering the disparity on the number of tests conducted. Since the study is randomised, it would be critical to allocate interventions based on the sufficient number of subjects that could not be manipulated while also ensuring that both known and unknown confounding variables are evenly distributed. Admittedly, the authors know that there is selection bias evident.
Another issue refers to the clinical validity of the period over which it has been measured (Cook and Campbell, 1979; Stewart et al, 2008). For people who do not have a background on spirometry and its application, feasibility and quality, it could be very difficult to distinguish the significance of the study based on planned comparisons of the number of tests performed and the proportions of the eligible target group at risk of COPD in a period of 6 months. It would be much harder, nonetheless, on the part of appraisers to determine the degree of efficacy of spirometer based on the fact that opportunistic testing was done in 5 minutes for TN model but with no information given for UC model. Because the study is an open trial, one can figure that it would have been more credible if the authors could have tabulated the symptoms relating to COPD or perhaps what makes a patient eligible instead of the reasons for decline or acceptance for the purpose of addressing clinical validity. It is said in here that this should not be neglected, what the paper is trying to argue is that outcomes must be embedded on the comparisons of the effectiveness of TN and UC models. UC model, because it adhered into a specific protocol will basically lack in such a basis.
After practices were recruited, the authors made use of a random numbers table whereby practices are randomised to models for delivery of spirometry to patients in the target group at risk of COPD. The practices were recruited through a newsletter distributed in all practices Southern Tasmania to which 6 urban and 2 rural practices responded. Age, which is over 35 years, and ever smoked regularly are used as inclusion criteria, no exclusion criteria were detailed, making the criteria as vague. Vague due to the fact that the study did not point out any gender criterion but in Table 1 it reflects the percentage of male smokers in both TN and UC model. Assuming, that majority of the smokers are male still the study should have determined the characteristics of patients belonging to the target group based on gender. Evidence concerning the effect of the intervention is equivocal because of this reason. Nevertheless, the authors noted that both model conform to similar profiles for age, gender, current smoker status and smoking pack-year history.
The pragmatic nature of health services trials depended heavily on completion of follow up (Sale and Kevin, 2004; Dixon-Woods, n.d.). Intention to treat analysis indicates that the patients analysed in the groups to which they were randomised will still be analysed based on their original group. True enough, there are 190 (84%) patients who have consulted a GP by 3 months after spirometry in the TN model. There are 11, 9 and 19 new doctor-recorded airflow obstructions for participants without prior self-reported COPD, new doctor-recorded airflow obstruction and for participants with prior self-reported diagnosis of COPD. Somewhat typically, the details of the total number of patients diagnosed belonging to UC model are not given. There are, however, 2 participants with airflow obstruction received a new doctor-recorded diagnosis of COPD and 4 demonstrated airflow obstruction among those with a prior self-reported COPD diagnosis.
Statistical significance was set at the 5% level. The authors noted, however, that there was a significance difference between proportions of the estimated eligible target group population that underwent spirometry in practices in the TN model (58.7%) versus in practices in UC model (7.7%) overall. Such a difference to reach the clinical significance is debatable especially that the models are not treated equally. Of concern are the spirometry quality and the impact of spirometry on diagnosis of COPD and cost of spirometry in TN model. The evident imbalance could mean that the magnitude of effect would be even greater for the UC model because the imbalance does not favour the UC model. But in the case that the imbalance favours the TN model, then it will confound the study. The perceptions of the GPs best manifest this wherein GPs valued high quality spirometry and increased testing of patients at risk of COPD in the TN model.
The study has a virtue of being conducted in Australian healthcare context particularly in primary care setting. However, variations could exist in terms of where specifically the study will be conducted. One cannot assure that the competence of nurses and care assistants in conducting spirometer testing, and not the effectiveness of spirometer, as well as the competence of GPs in interpreting would be consistent among all the population. Yes, applicability is possible but one cannot ignore the fact that it could also be considered reductio ad absurdem. As the study could not confirm its universality, the authors maintain that the findings may not be generalisable to all primary practice. The burden of proof lies in the specificity of the setting and the interventions which make the study less applicable. As mentioned earlier, one of the inherent problems of the study is the exclusion of a larger group of eligible patients especially that the authors did not provide rationales of conducting the study in Southern Tasmania. Even so, a valid conclusion would be a certain subgroup could benefit from implementation of a local health policy based on the evidences presented in the study if the whole population would not benefit from the said study.
References
Caracelli, V J and Greene, J C 1997, ‘Crafting mixed-method evaluation design,’ in J C Greene and V J Caracelli (eds.), Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms, New Directions for Program Evaluation, No. 74., Jossey-Bass, San Francisco, CA, pp. 19-32.
Cook, T D and Campbell, D T 1979, ‘Validity,’ in T D Cook and D T Campbell. Quasi-experimentation: Design and analysis for field settings, Houghton Mifflin, Boston, MA, pp. 37-94.
Critical appraisal and using the literature, retrieved on 5 March 2009, from http://www.shef.ac.uk.
Dixon-Woods, M n.d., Integration of research based of mixed-methods, Department of Health Sciences, University of Leicester.
Evaluation tool for mixed method studies, HCPRDU, retrieved on 5 March 2009, from http://www.fhsc.salford.ac.uk.
Greene, J C and Caracelli, V J 1997, ‘Defining and describing the paradigm issue in mixed-method evaluation,’ In J C Greene and V J Caracelli (eds.), Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms. New Directions for Program Evaluation, No. 74, Jossey-Bass, San Francisco, CA, pp. 5-18.
Miller, M R, Crapo, R and Hankinson, J 2005, ‘General considerations for lung function testing,’ European Respiratory Journal, vol. 26, no. 1, pp. 153-161.
Sale, J E M and Brazil, K 2004, ‘A strategy to identify critical appraisal for primary mixed method studies,’ Quality and Quantity, vol. 38, no. 4, pp. 351-365.
Stewart, M, Makwarimba, E, Barnfather, A, Letourneau, N and Neufeld, A 2008, ‘Researching reducing health mixed method approaches disparities,’ Social Science and Medicine, vol. 66, no. 6, pp. 1406-1417.
Sydenstricker-Neto, J, 1997, Research Design and Mixed-Method Approach – A Hand-on Experience.
Credit:ivythesis.typepad.com
0 comments:
Post a Comment