See Lipkovich et al (2017) [6] for a comprehensive review of data-driven methods to define subgroups using clinical trials data. PS was responsible for writing and critical review on manuscript. During the review process of this manuscript, the authors had been asked to provide the data based on the pre-specified definitions. 2. These criteria will help clinicians deciding whether to use subgroup analyses to guide their patient care. On Enrichment Strategies for Biomarker Stratified Clinical Trials. FF: Nothing to disclose. sharing sensitive information, make sure youre on a federal Cook DI, Gebski VJ, Keech AC. power.analysis.subgroup . PubMedGoogle Scholar. NSTE-ACS subgroup of the PLATO trial[1]. The research work was partially supported by P01CA142538 (XFW) and R01AG066883 (XFW), and P30CA15083 (Mayo Clinic Comprehensive Cancer Center Grant; JLR and SJM). SJG: Consultant: Allergan, Apellis, Bausch and Lomb, Boehringer Ingelheim, Johnson and Johnson, Kanaph; Research funds: American Academy of Ophthalmology, Apellis, Boehringer Ingelheim, NGM Bio, Regeneronunrelated to this study. of medical sciences, Cardiology; and Uppsala clinical research center (UCR), Uppsala Universit, Long-term risk of atrial fibrillation or flutter after transcatheter patent foramen ovale closure: a nationwide Danish study, Cationic proteins from eosinophils bind bone morphogenetic protein receptors promoting vascular calcification and atherogenesis, Dynamical improvement after rescue balloon angioplasty for pulmonary vein occlusion complicated with radiofrequency catheter ablation, Lower income, higher risk: disparities in treatments and outcomes of patients with acute myocardial infarction, COVID-19 vaccination-related myocarditis: a Korean nationwide study. 2023 American Medical Association. Such data sets may be very consistent. Understanding of interaction (subgroup) analysis in clinical trials Milos Brankovic, Milos Brankovic orcid.org/0000-0002-3996-0813 Clinical Epidemiology Unit, Department of Cardiology, Erasmus Medical Center, Rotterdam, The Netherlands School of Medicine, University of Belgrade, Belgrade, Serbia Search for more papers by this author Treating individuals 2. Was there any indirect evidence to support the apparent subgroup effectfor example, biological rationale, laboratory tests, animal studies? Cmaj. Adolescent Psychological Assets and Cardiometabolic Health Maintenance in Adulthood: Implications for Health Equity. Recent NI trials have performed a priori subgroup analyses using standard statistical tests for interaction, but there is increasing interest in more flexible machine . Meta-analysis evaluates the heterogeneity of treatment effects reported from multiple clinical studies and synthesizes the estimates of the treatment effects. padding-bottom: 0px; For example, the chance of making at least one false positive claim for conducting 10 repeated tests at 5% significance level is 40% when there is no treatment effect. Bmj. Varun Chaudhary. Subgroup Analysis - Statistics How To comment[2], we also agree that subgroup analyses, especially those If there exist differential treatment effect across subgroups, two types of treatment-by-subgroup interaction have been described -- quantitative versus qualitative interaction. FF was responsible for writing, critical review and feedback on manuscript. Comparison of Confirmatory versus Exploratory Subgroup Analyses. Ou F, Le-Rademacher JG, Ballman KV, Adjei AA, & Mandrekar SJ (2020). Zhang C, Zhang M, Qiu W, Ma H, Zhang X, Zhu Z, et al. How can we be certain that the data presented were not taken out of the context of an array of multiple analyses performed, chosen in the end because they best fit a hypothesis? Apparently compelling subgroup effects often prove spurious, and guidance is needed to differentiate credible from less credible subgroup claims. As a library, NLM provides access to scientific literature. This would indicate stability of a treatment effect over a broad study population. More information on meta-analysis can be found in Higgins et al (2019) [12]. Bmj 2003;326:219. Reviews of these more flexible methods, such as the fallback procedure [15] and MaST procedure [16], can be found in Matsui et al (2014) [17] and Dmitrienko et al (2017) [18]. 2007 Jul;3(7):407-13. We therefore provide 5 criteria to use when assessing the validity of subgroup analyses: (1) Can chance explain the apparent subgroup effect; (2) Is the effect consistent across studies; (3) Was the subgroup hypothesis one of a small number of hypotheses developed a priori with direction specified; (4) Is there strong preexisting biological supp. Med J Aust. The issue about invasive vs. non-invasive management is In the original publication , the a priori power calculation based on the primary outcome postcardiotomy ventricular . General guidance on exploratory and confirmatory subgroup analysis in late-stage clinical trials. examine outcomes in relation to intent of revascularization defined prior Thus, assessing treatment outcome heterogeneity across subgroups and identifying patient characteristics that may modify the effect of the intervention under investigation has become common practice [4]. Equal allocation yields the highest power. 2019;38:56573. 2011;342:d3527. PDF When to believe a subgroup analysis: revisiting the 11 criteria - Nature How to use a subgroup analysis: users' guide to the medical - PubMed In the meantime, to ensure continued support, we are displaying the site without styles A simple gating approach is to sequentially test for the overall treatment effect before evaluating the effect within subgroups and once the null hypothesis of the overall treatment effect fails to be rejected, no subgroup analysis should be conducted. PMID: Sun X et al. Was the subgroup effect consistent with evidence from previous related studies? Giannaccare G, Pellegrini M, Sebastiani S, Bernabei F, Roda M, Taroni L, et al. The first 4 criteria are applicable to individual studies or systematic reviews, the last only to systematic reviews of multiple studies. Sun X, Briel M, Busse JW, You JJ, Akl EA, Mejza F, et al. Which data will end up in a publication? Besides the size and nature of an interaction, the allocation ratio among subgroups also has an impact. Updating criteria to evaluate the credibility of subgroup analyses. The statistical term to describe differential treatment effects by subgroups is the interaction between the treatment and the subgroup variable. When common cutoffs are not available for a continuous biomarker, cutoff points can be identified by data driven approaches such as simple percentiles (e.g. In such cases, a favorable treatment effect in a targeted subgroup is anticipated whereas the treatment would likely have no effect or even a deleterious effect in the complementary subgroup, or in the unselected population. Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. This allows for full transparency of the two data sets, based on the pre-specified vs. the post-hoc cut-off values. Although subgroups can be defined using continuous variables, the loss of power and efficiency from categorization of continuous variables should be considered when defining subgroups in this manner [7]. In RCTs, subgroups are often defined by demographic variables, such as age, sex and race. during the initial 10 days; and this time-point is also closer to the end my editorial. Jama 2019;321:188094. 2012 Mar 15;344:e1553. A pooled analysis of two RCTs of 107 patients with highly relapsing neuromyelitis optica spectrum disorder [18] illustrates effective adherence to these principles in its design. Key components of our proposed approach include targeted literature searches and key informant interviews to identify the most important subpopulations a priori during topic scoping, a framework for assessing the credibility of subgroup analyses reported in studies, and structured investigation of sources of heterogeneity of intervention effects. When reported, this information can often be found in the methods section of the article. A subgroup analysis of PLATO patients who underwent PCI has long been awaited, but has not been presented yet. Spinal manipulative therapy (SMT) is a guideline-recommended treatment option for spinal pain. If subgroups are defined by continuous variables, it is preferable to use well established or published cutoffs. Therefore, pre-specification of subgroups could be made mandatory for publication. Ophthalmology 2020;127:S135s45. treatment decisions (which we acknowledge). They could indicate a differential treatment effect based on age, but could also be play of chance. so were not immune to the pitfalls of post-hoc analyses I had outlined in MB: Research funds: Pendopharm, Bioventus, Acumedunrelated to this study. This pre-specified analysis has been reported In general, when a specified level of overall type I error rate is allocated to testing multiple hypotheses with respect to different patient subgroups or endpoints and each test is conducted at its allocated significance level, the overall type I error rate will not exceed its specified level regardless if these tests are dependent or not. 2023 May;49(5):545-553. doi: 10.1007/s00134-023-07066-z. Subgroup analysis: What it is + How to avoid mistakes Archives of Neurology & Psychiatry (1919-1959), How to Use a Subgroup Analysis with Dr Gordon Guyatt, JAMAevidence: The Rational Clinical Examination, JAMAevidence: Users' Guides to the Medical Literature, JAMA Surgery Guide to Statistics and Methods, Antiretroviral Drugs for HIV Treatment and Prevention in Adults - 2022 IAS-USA Recommendations, CONSERVE 2021 Guidelines for Reporting Trials Modified for the COVID-19 Pandemic, Global Burden of Skin Diseases, 1990-2017, Guidelines for Reporting Outcomes in Trial Protocols: The SPIRIT-Outcomes 2022 Extension, Mass Violence and the Complex Spectrum of Mental Illness and Mental Functioning, Organization and Performance of US Health Systems, Spirituality in Serious Illness and Health, The US Medicaid Program: Coverage, Financing, Reforms, and Implications for Health Equity, Screening for Prediabetes and Type 2 Diabetes, Statins for Primary Prevention of Cardiovascular Disease, Vitamin and Mineral Supplements for Primary Prevention of of Cardiovascular Disease and Cancer, Statement on Potentially Offensive Content, Register for email alerts with links to free full-text articles. Dept. Med J Aust. PMC Outcomes were identified through a priori subgroup analyses, and dose-responses were assessed for exercise intensity and exposure duration. Teprotumumab for patients with active thyroid eye disease: a pooled data analysis, subgroup analyses, and off-treatment follow-up results from two randomised, double-masked, placebo-controlled, multicentre trials. particular subgroup analysis rather than the previously announced, pre- The recommendation is based on multiple systematic reviews. ISIS-2 (Second International Study on Infarct Survival) Collaborative Group, Randomized trial of intravenous streptokinase, oral aspirin, both or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2, Statement: updated guidelines for reporting parallel group randomised trials, Analysis and interpretation of treatment effects in subgroups of patients in randomized trials, Credibility of claims of subgroup effect in randomized controlled trials: systematic review, Comparison of ticagrelor, the first reversible oral P2Y12 receptor antagonist, with clopidogrel in patients with acute coronary syndromes: rationale, design, and baseline characteristics of the PLATelet inhibition and patient Outcomes (PLATO) trial, ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation, Ticagrelor versus clopidogrel in patients with ST-elevation acute coronary syndromes intended for reperfusion with primary percutaneous coronary intervention: a Platelet Inhibition and Patient Outcomes (PLATO) Trial subgroup analysis. Is a subgroup effect believable? https://doi.org/10.1038/s41433-022-01948-0, DOI: https://doi.org/10.1038/s41433-022-01948-0. A priori power calculator for subgroup contrasts power.analysis Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors. During the index admission, 61.0% of the overall population underwent percutaneous coronary intervention (PCI) and 4.5% underwent coronary artery bypass grafting (CABG). See Ondra et al (2016) [22] for examples and reviews of confirmatory subgroup analysis in cancer clinical trials that test targeted agents and immunotherapy. Simultaneous subgroup analyses create multiplicity, inflating the defined nominal significance level (alpha) [10] which increases the likelihood of spurious and compelling results by chance alone [1]. center (UCR), from randomization, the larger potential for such time-dependent Preference cookies are used to store user preferences to provide content that is customized and convenient for the users, like the language of the website or the location of the visitor. Subgroup Analysis Advantages. The https:// ensures that you are connecting to the Epub 2022 Nov 17. With regard to the published article, it should be emphasized that Subgroup by treatment interaction assessments are routinely performed when analysing clinical trials and are particularly important for phase 3 trials where the results may affect regulatory labelling. Common challenges with subgroup analyses include poor definitions, low statistical power, and inflated type I error due to multiple hypotheses testing. Bretz F, Maurer W, Brannath W, & Posch M (2009). PLATO had allowed randomization of patients with acute coronary syndromes regardless of whether aiming for an invasive or non-invasive strategy, or if a revascularization procedure had been intended or actually performed. 2020 Jun 1;30(6):839-845. doi: 10.1093/icvts/ivaa042. Quality of reporting of key methodological items of randomized controlled trials in clinical ophthalmic journals. Syed MA, Aiyegbusi OL, Marston E, Lord JM, Teare H, Calvert M. Geroscience. Using the same definition of subgroup enables comparison of outcomes between similar subgroups across different studies/clinical trials. Although knowing how well a new treatment works in patients with a specific biomarker or a specific combination of disease stage and histology are important for the patients and their clinicians to make informed treatment decisions, it is important to be cautious when interpreting the results of subgroup analyses. Dmitrienko A, Muysers C, Fritsch A, & Lipkovich I (2016). Research Methods & Reporting - Jstor Advertisement intended for healthcare professionals, Vivantes Auguste-Viktoria-Klinikum, Klinik fr Innere Medizin Kardiologie, Diabetologie und konservative Intensivmedizin, Rubensstr. present results regarding actually performed revascularization in patients Brand KJ, Hapfelmeier A, Haller B. Clin Trials. by dr. Helmut Schuehlen. Subgroup analyses in clinical trials and observational studies have been discussed in regulatory agency guidelines [13] and comprehensive review of this topic have been published in applied statistical journals [45]. In confirmatory cases, the subgroups must be clearly defined and the endpoints delineated with a small number of hypotheses about subgroup-specific treatment effects. Lancet Neurol. Safety and efficacy of tocilizumab versus azathioprine in highly relapsing neuromyelitis optica spectrum disorder (TANGO): an open-label, multicentre, randomised, phase 2 trial. Clinicians, when trying to apply trial results to patient care, need to individualize patient care and, potentially, manage patients based on results of subgroup analyses. In this example, EGFR mutation is a predictive biomarker for gefitinib in NSCLC. Subgroup analyses, either specified a priori or post-hoc, are performed to assess the treatment effect specific to a subgroup of treated patients. Dmitrienko A, Millen B, & Lipkovich I (2017). The opinions expressed in this article are not necessarily those of the Editors of the European Heart Journal or of the European Society of Cardiology. pool.groups: Pool the results of two treatment arms; power.analysis: A priori power calculator; power.analysis.subgroup: A priori power calculator for subgroup contrasts was, however, substantial crossover. In the development of cancer targeted agents and immunotherapy, RCTs are conducted with an interest in demonstrating the treatment effect in all patients and/or in a targeted subgroup. Schematic plot of qualitative interaction between treatment and EGFR mutation observed in IPASS trial. The https:// ensures that you are connecting to the They may find out that certain subgroups respond better than others or that there are differences between men and women, for example. PubMed Central The publisher's final edited version of this article is available free at, Statistical principles for clinical trials (ICH E9): an introductory note on an international guideline, Enrichment Strategies for Clinical Trials to Support Approval of Human Drugs and Biological Products, Guideline on the investigation of subgroups in confirmatory clinical trials. SJG was responsible for critical review and feedback on manuscript. Internet Explorer). One possible explanation was the predominant vegetarian diet and low intake of omega-3 fatty acids in India. Their credibility is compromised by the effect of intervention and lack of statistical power [7, 9]. Careers, Unable to load your collection due to an error. A priori power calculator for subgroup contrasts Description This function performs an a priori power estimation for a test for subgroup differences within a meta-analysis. A systematic review and meta-analysis comparing First of all, subgroup analyses may demonstrate consistent results over various complementary subpopulations, e.g. Long-term cardiometabolic health in people born after assisted reproductive technology: a multi-cohort analysis. Google Scholar. These circumstances carry important therapeutic consequences. Accessibility Statement, Our website uses cookies to enhance your experience. The former is credible if planned based on a prespecified hypothesis, if there is a justified direction of the overall and subgroup effect, and if there is appropriate statistical testing for the underlying hypothesis. PMID: Study to Prospectively Evaluate Reamed Intramedullary Nails in Tibial Fractures (SPRINT) Investigators. 2004;180:28991. So, statisticians would caution us that these analyses are hypothesis generating at best, and cannot be regarded as evidence. Here, we revisit the 11 criteria (Table1) introduced by Sun et al. Updating criteria to evaluate the credibility of subgroup analyses. As a service to our customers we are providing this early version of the manuscript. Effect of dexmedetomidine versus lorazepam on outcome in Manchikanti L, Datta S, Smith HS, Hirsch JA. About one fifth of those planned for The former is credible if planned based on a prespecied hypothesis, if Lipkovich I, Dmitrienko A, DAgostino RB (2017). When there is sufficient statistical evidence to indicate an interaction, it is important to state the treatment effects by subgroups and not use the overall average treatment effect on all patients to characterize the effect of treatment across all subgroups. These criteria will help clinicians deciding whether to use subgroup analyses to guide their patient care. ", M.D., Dept. A priori subgroup analyses revealed significant association between body mass index and treatment differences (=0.95 mm Hg (95% CI=1.56, 0.34); P=0.007). The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. confirm heterogeneity of treatment effect across subgroups, explore the possibility of treatment effect heterogeneity across subgroups, carefully chosen multiple testing procedure (MTP) for proper overall type I error rate, unclear, sometimes impossible to interpret correctly. At the end of the trial, 64.3% had undergone PCI and 10.2% CABG. median), or visualization with statistical graphs. For example, are the subgroups plausible given what is known about the clinical, pharmacological, and biological mechanisms? Post-hoc subgroup analyses, in contrast, are data driven and are considered exploratory or hypothesis generating. 2009;52:51522. government site. We decided to perform the analysis with the cut-point of 10 days Subgroup Analysis LITFL CCC Research When heterogeneity is not significant, pooling the information from multiple studies is preferred as it usually increases power. A statistical primer on subgroup analyses For example, it is common in oncology to use age cutoffs of 40 years and 65 years to classify patients into age groups: <40 as adolescent and young adult, 40-65 as adult, and >65 as older adult. DL: institutional research grants and lecture fees from AstraZeneca. Subgroup analysis in clinical trials. Careers. National Library of Medicine Ranibizumab and bevacizumab for neovascular age-related macular degeneration. Study Group, Association Between Visual Acuity and Fluid Compartments with Treat-and-Extend Intravitreal Aflibercept in Neovascular Age-Related Macular Degeneration: An ARIES Post Hoc Analysis, Research Methodology for the Ophthalmologist, Cancel volume36,pages 20752077 (2022)Cite this article. The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: systematic review. A trial database is easily analysed for all these different cut-off values. Using network meta-analyses, we aim to investigate which SMT "application procedures" have . Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. male and female, young and elderly patients. This article reviews key statistical concepts associated with planning, conducting, and interpreting subgroup analyses in RCTs. effect of ticagrelor for invasive vs. non-invasive management is to It is mandatory to procure user consent prior to running these cookies on your website. Google Scholar. However, these reviews fail to consider that clinical effects may depend on SMT "application procedures" (i.e., how and where SMT is applied). The estimates of treatment-by-subgroup interactions from these models can be interpreted as the differential treatment effects, i.e. } 2020;19:391401. The Author 2014. Farrokhyar, F., Skorzewski, P., Phillips, M.R. Arguably, the consistency of the subgroup effects in subsequent well-designed trials provide stronger credibility. 2023 Jan 17;12(2):e026173. -7. MRP was responsible for conception of idea, critical review and feedback on manuscript. Cornea. [7] and provide literature case examples to illustrate important principles and concepts in the interpretation of a subgroup analysis and to guide researchers in deciding their credibility in the ophthalmological literature. The Bonferroni approach is easy to implement but could be overly conservative, as it ignores the correlation between outcomes of patients in the subgroups and those of the overall patient group. More discussion on prognostic and predictive biomarkers in cancer research can be found in Mandrekar and Sargent (2009) [10] and Ballman (2015) [11]. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet. Undefined cookies are those that are being analyzed and have not been classified into a category as yet. Ann Intern Med. What is the interest of the scientific community in performing subgroup analyses? As illustrated in Figure 2, gefitinib (an EGFR inhibitor) was associated with significantly better progression-free survival (PFS) compared to control (carboplatin plus paclitaxel) in EGFR mutants, while gefitinib was associated with significantly worse PFS compared to control in EGFR wild types. Rothwell PM. Understanding of interaction (subgroup) analysis in clinical trials The effect of a treatment may vary by baseline patient characteristics (e.g. Conflicts of interest: H.S. See Wang et al (2018) [21] for further discussion of strategies to optimize the power of treatment-by-subgroup interactions in biomarker-stratified clinical trials. Accessibility ACS management and PLATO. 2011 Mar 28;342:d1569. A systematic review of subgroup analyses in randomised clinical trials Power of Statistical Tests for Subgroup Analysis in Meta-Analysis While the first two approaches are performed frequently, the third is considered inappropriate by the scientific community. Tutorial in Biostatistics: Data-driven subgroup identification and analysis in clinical trials, The cost of dichotomising continuous variables. Finally, in trials with an overall negative result, subgroup analyses might identify patient subsets with a significant treatment effect. A table or figure reporting about a dozen subgroup analyses is a near ubiquitous feature of major clinical trial publications. 2008;5:e230. management without revascularization ultimately underwent PCI[3]. 2017;43:162. For instance, the CATT [14] conducted a non-inferiority trial to compare the efficacy of ranibizumab versus bevacizumab on either a monthly scheduled or an as needed regimen in patients with neovascular age-related macular degeneration and found equivalent gain in VA by treatment and dosing regimen at 1 year. In qualitative interactions, the treatment benefit is in favor of the experimental treatment in one subgroup but is unfavorable or neutral for the other subgroup. While we agree that these analyses are hypothesis generating due to Ophthalmic Epidemiol. Unlike confirmatory subgroup analysis, exploratory subgroup analyses are intended to obtain preliminary evidence and to generate hypotheses for future investigation. government site. Before (i.e. The judicious design, analysis, and reporting of RCTs allow surgeons to effectively use the results in routine practice [1,2,3]. Multiple subgroup analyses of the same data increase the risk of generating false positive findings. The site is secure. clear: left; Given differences in the administration of surgical treatments and extent of biological variability, the interaction between treatment effect and various patient variables should be interpreted with caution [1]. the possible influence of post-randomization decisions, they were chosen Ill-defined and unreproducible subgroups could lead to biased and unreproducible estimates of treatment effects and hard-to-interpret findings. Methods: National Library of Medicine Additional factors that affect the validity and the reproducibility of subgroup analyses are missing data and measurement error associated with the variables for searching and defining subgroups.