The Journal of Bone & Joint Surgery

The Journal of Bone & Joint Surgery, Volume 94, Issue Suppl 1(E)

Scientific Articles | July 18, 2012

Efficient Designs: Factorial Randomized Trials

Daniel B. Whelan, MD, MSc, FRCSC¹; Katie Dainty, PhD¹; Jaskarndip Chahal, MD, FRCSC¹

¹ University of Toronto Sports Medicine Program, Division of Orthopaedic Surgery (D.B.W. and J.C.) and the Li Ka Shing Knowledge Institute (K.D.), St. Michael’s Hospital, 55 Queen Street East, Suite 800, Toronto, ON, M5C 1R6, Canada. Email address for J. Chahal: jaschahal@hotmail.com

View Disclosures and Other Information

Disclosure: None of the authors received payments or services, either directly or indirectly (i.e., via his or her institution), from a third party in support of any aspect of this work. None of the authors, or their institution(s), have had any financial relationship, in the thirty-six months prior to submission of this work, with any entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. Also, no author has had any other relationships, or has engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.

Disclosure statement for author(s): PDF

Investigation performed at St. Michael’s Hospital and the Li Ka Shing Knowledge Institute, University of Toronto, Toronto, Ontario, Canada

J Bone Joint Surg Am, 2012 Jul 18;94(Suppl 1(E)):34-38. doi: 10.2106/JBJS.L.00243

5 Recommendations (Recommend) | 3 Comments | Saved by 3 Users Save Case

Article

Comments (0)

text A A A

Abstract

Abstract:

As an alternative to conducting multiple parallel group randomized controlled trials, the factorial design is an efficient means of evaluating multiple interventions. This review highlights when it is appropriate to conduct a factorial trial, considers the structure and nomenclature of factorial designs, and discusses various methodological considerations, including sample size calculation, the role of statistical interactions between different interventions, and statistical power. Several pertinent examples in orthopaedics and medicine are used to highlight the key concepts.

Figures in this Article

Introduction

The evaluation of multiple interventions in medicine can be achieved by conducting multiple traditional parallel group randomized controlled trials (RCTs). A more efficient design alternative, however, exists in the factorial RCT. In the surgical literature, the mainstay to date has been the parallel group trial, which is the optimal design when two interventions are mutually exclusive in that they cannot be used in conjunction with one another¹. An example would be the comparison of outcomes following open versus arthroscopic shoulder stabilization for recurrent anterior shoulder instability. Although both of these procedures are performed to achieve the same outcome, they cannot be performed together or in succession. In contrast, there are some interventions that can be performed together with the common goal of achieving a similar outcome¹. For example, in an ongoing investigation of different irrigating pressures and irrigating solutions for the management of open fracture wounds (the fluid lavage of open wounds [FLOW] study), investigators compare two different solutions with three different lavage pressures with the common goal of decreasing reoperation rates². To compare all permutations of fluid/pressure combinations with a traditional parallel group design, one would have to conduct up to fifteen separate RCTs (Fig. 1). The financial and time-related costs associated with such an endeavor constitute a major challenge regarding funding and feasibility. To avoid such limitations in feasibility, the FLOW study, to the best of our knowledge, is the first trial in orthopaedic trauma surgery to utilize a factorial design², one that is ideally suited to investigate multiple interventions that are not mutually exclusive and that aim to achieve the same outcome^1,3-7.

Fig. 1

Illustration of fifteen possible permutations of solutions and lavage pressures in the FLOW study². Arrows highlight comparisons within a factor, and straight lines highlight comparisons between factors.

View Large | Download Slide(.ppt)
Add to My JBJS

Fisher was the first scientist to use the term “factorial” in print in 1935 in his book, The Design of Experiments⁸. He argued that “complex designs” such as factorial designs were more efficient than studying one factor at a time⁹. The utilization of factorial RCTs has increased from 1% of all trials on MEDLINE (1970 to 1980) to 4.7% (1993 to 2003)⁵. Despite this increase in the general medical literature and more specifically in the field of internal medicine and cardiology⁵, to the best of our knowledge, their use in orthopaedics has certainly been limited^2,10. Nevertheless, factorial trials present an attractive option to orthopaedic investigators who wish to capitalize on efficiency to optimize the use of time, financial resources, and smaller patient populations for less common conditions.

Structure of a Factorial Trial

Each intervention to be studied is called a “factor.” Each factor also has associated mutually exclusive treatments called “levels.” The number of factors and levels can be selected to best reflect the clinical question being asked. In the simplest form of factorial design, there are two factors and two levels; the nomenclature for this is a 2 × 2 factorial design.

Understanding the nomenclature scheme for a factorial trial can be confusing initially. If an investigator wishes to look at two factors called “Y” and “Z,” then the general nomenclature for this factorial design would be Y × Z. If intervention Y has four levels and intervention Z has three levels, the final nomenclature would be a 4 × 3 design.

Examples of more complex factorial designs would include a 2 × 3 or a 3 × 3 × 3 trial. In the latter example, there are three interventions (factors) of interest, each with three associated levels of treatment. In the FLOW study, the authors are investigating the effect of two factors: one with two levels (solution type) and one with three levels (lavage pressures). Based on the aforementioned nomenclature scheme, this constitutes a 2 × 3 factorial design² (Fig. 2).

Fig. 2

Factorial design used in the FLOW study².

View Large | Download Slide(.ppt)
Add to My JBJS

Design Considerations

Sample Size

When calculating sample size in a factorial design, one must first calculate the sample size for each intervention based on the target effect size^5,6. The factorial trial sample size is then simply the larger of these calculations. The trial is then said to be powered to detect the main effects of each intervention. Provided that the trial is adequately powered to detect differences for the larger trial sample, it will automatically meet the sample size requirement for the second intervention under study.

For example, in the FLOW study², the sample size was chosen to identify if there is any difference in the pairwise comparisons of the three irrigation pressure groups on reoperation at twelve months. This sample size also allows this study to establish if there is a difference between soap and saline solutions on the same outcome. Multiple calculations are necessary, and therefore, it is important to adjust the calculation accordingly. The FLOW investigators took into account the three pairwise comparisons by using the Tukey method, which is a single-step multiple-comparison procedure often used in conjunction with an analysis of variance (ANOVA) to find which means are different from one another². This method is less conservative than the Bonferroni correction, which is also used for decreasing the type I error that results from multiple comparisons.

The Role of Interactions

A concern with the factorial design is the possibility of the existence of interaction among the factors under study and its potential impact on sample size^3,7. Interaction means that the effect of intervention Y differs depending on the presence or absence of intervention Z, or vice versa. If there are no interactions between interventions, a modest increase in sample size can allow for two experiments to be conducted almost “for the price of one.” This sample size would be much less than the sum of two independent trials³. To detect a small interaction, a large increase in sample size is required, whereas a much smaller increase in sample size is required to detect a larger interaction^5,6. However, interaction may not play a large role since the incidence of interaction in factorial trials has been estimated to be approximately 6%⁵.

Interactions can be classified as either qualitative or quantitative^1,5. For qualitative interactions, the direction of the observed effect is altered because of the interaction. In other words, each intervention has an effect, but the interaction of the two interventions results in an effect in the opposite direction. For example, in a factorial trial investigating the role of providing an educational booklet and physician advice in managing back pain, it was demonstrated that while providing either the booklet or verbal advice alone resulted in an improvement in function and back pain, there was no significant difference in these parameters if both interventions were implemented together¹⁰. The findings of this study supported previous evidence that detailed information booklets may not reinforce verbal advice.

In the case of quantitative interactions, only the magnitude of the difference is altered because of the interactions^1,5. There are two types of quantitative interactions: synergistic and antagonistic. A synergistic relationship is one that is more effective than the simple addition of the effects of individual interventions (i.e., 1 + 1 > 2). An antagonistic interaction is one that is less effective than the addition of the effects of individual interventions (i.e., 1 + 1 = 1.01 to 1.99)^1,5. For example, in the FLOW study, if it was observed that the additive effects of high-pressure lavage and the use of soap solution on reoperation rates were greater than the sum of the individual improvement in reoperation rates, this phenomenon could be explained by the presence of a synergistic interaction.

Montgomery et al.⁶ have suggested that when there may be a qualitative interaction between groups, the trial should be powered to detect the interaction. In contrast, in the presence of a possible quantitative interaction, the trial should be powered to detect the main effects.

McAlister et al.⁵ have illustrated the concept of the interaction ratio, which compares the effects of each treatment in the presence and absence of the other treatment. Ratios above 1.0 indicate synergy, while ratios below 1.0 indicate antagonism⁵. Ratios greater than 1.25 or less than 0.8 have been defined as being clinically substantial. For example, in a factorial trial comparing early invasive (coronary angiography ± revascularization within seven days from admission) to noninvasive medical treatment (treatment A) with dalteparin or a placebo (treatment B) in patients with acute coronary syndrome¹¹, there is a nonspecific trend toward synergy between the treatments (interaction ratio: 1.06; 95% confidence interval, 0.67 to 1.69). The fact that this observed ratio was statistically nonsignificant is not surprising given that most factorial trials are likely underpowered.

Often, prior to commencing a factorial trial, there is much uncertainty as to whether there will be interactions. Bayesian methods, which measure probability as a “state of knowledge” and are an extension of logic theory, might be helpful to formalize such prior information or beliefs⁶. Furthermore, few factorial trials are powered a priori to detect interactions⁷. Once again, the smaller the interaction, the larger the sample size that is required to detect it. This is where the efficiency of choosing a factorial design can be lost. However, even when there is an interaction, it may not affect the way the final results of the study are interpreted. In their review of published factorial trials, McAlister et al. found only two cases (6%) where there was a substantial interaction, and in only one of these cases did this lead to a qualitative difference in interpretation in the final results of the study⁵. Finally, interactions are more likely to occur with behavioral interventions⁶, and their role in surgical interventions requires more elucidation.

In the FLOW study², the interaction of soap with pressure will be examined post hoc by including the main effects and their interaction terms in the Cox regression with the outcome variable as reoperation. Any trends identified with interaction analyses can give rise to future research questions.

Statistical Analysis and Power Considerations

For an illustration of statistical analysis, an example of a factorial trial is presented in Table I. This example details a hypothetical study looking at the operative versus nonoperative treatment of small rotator cuff tears (treatment A) and the use of platelet-rich plasma (PRP) or placebo (treatment B) in this same patient group.

View Larger

TABLE I A 2 × 2 Factorial Design for a Study Investigating the Role of Surgical Rotator Cuff Repair and the Use of Platelet-Rich Plasma for the Treatment of Small (<1 cm) Rotator Cuff Tears*

Surgical repair of small rotator cuff tear (A)	Platelet-Rich Plasma (B)
Yes	No	Margin
Yes	Cell AB [n = 100]	Cell A0 [n = 100]	All patients treated with RCR (cells AB + A0) [n = 200]
No	Cell B0 [n = 100]	Cell 00 [n = 100]	All patients not treated with RCR (cells B0 + 00) [n = 200]
Margin	All patient receiving PRP (cells AB + B0) [n = 200]	All patients not receiving PRP (cells A0 + 00) [n = 200]	n = 400

*Sample size (n = 400) based on target difference of 0.39 SDs, 90% power, and 1% alpha. RCR = rotator cuff repair, PRP = platelet-rich plasma, and SD = standard deviation. This table is based on information from McAlister et al.⁵

Add to My JBJS

TABLE I A 2 × 2 Factorial Design for a Study Investigating the Role of Surgical Rotator Cuff Repair and the Use of Platelet-Rich Plasma for the Treatment of Small (<1 cm) Rotator Cuff Tears*

Surgical repair of small rotator cuff tear (A)	Platelet-Rich Plasma (B)
Yes	No	Margin
Yes	Cell AB [n = 100]	Cell A0 [n = 100]	All patients treated with RCR (cells AB + A0) [n = 200]
No	Cell B0 [n = 100]	Cell 00 [n = 100]	All patients not treated with RCR (cells B0 + 00) [n = 200]
Margin	All patient receiving PRP (cells AB + B0) [n = 200]	All patients not receiving PRP (cells A0 + 00) [n = 200]	n = 400

The two basic approaches to analysis include the “at the margins” and “inside the table” approach⁵. In the absence of interactions, the primary analysis of a factorial trial is performed “at the margins” of the 2 × 2 table in which the effect of each factor is tested by pooling across the other^5,12. In reference to Table I, the efficacy of surgery for rotator cuff repair is determined by comparing outcomes among all patients treated with rotator cuff repair (AB + A0) with those of all patients treated nonoperatively (B0 + 00). Similarly, the efficacy of PRP injection is determined by comparing outcomes among all patients treated with injections (AB + B0) with those not treated with injections (A0 + 00). It is important to note that the outcome measures should not be analyzed and presented separately for each of the four factorial cells in the case of a 2 × 2 factorial design (i.e., an “inside the table” analysis)⁶. An attempt to do this would not allow the investigators to realize the full efficiency and purpose of the factorial design⁶. By performing analyses at the margins, investigators can obtain evidence about the efficacy from far fewer patients than would be needed if A and B were individually tested in two separate trials⁵.

If the investigators of our hypothetical study suspected there was an interaction occurring between surgery and the PRP injection, then it would be necessary to conduct the analysis using the “inside the table” approach⁵. The efficacy of surgery would be determined by comparing outcomes of patients who receive only rotator cuff surgery (cell A0) with those patients who receive neither surgery nor PRP injection (cell 00). Similarly, the efficacy of PRP injections is determined by comparing cell B0 with cell 00⁵. Since the latter two comparisons only include patients in two of the four cells in the 2 × 2 table, the “effective” sample size will be lower and will result in the reduction of power if appropriate revisions in sample size are not made¹. By performing both types of analysis, an interaction ratio can be calculated, and trends toward synergy and antagonism can be identified.

In a parallel group RCT, the power to detect treatment differences depends on the number of participants in the individual groups being compared rather than the total number in the trial. This represents an inefficient use of resources because the power is determined by approximately half of the total sample size⁶. In a factorial design, it is possible to simultaneously investigate the effect of two interventions by including all participants in both analyses⁶. In contrast to the parallel group design, the entire sample size is used to calculate the effectiveness of each intervention in the factorial design. Using our example in Table I, assume there are 400 patients enrolled in this factorial trial (sample size based on target difference 0.39 standard deviations [SDs], 90% power, 1% alpha). Each comparison would involve all 400 patients (200 versus 200) but makes the assumption that interventions do not interact. However, to conduct two separate parallel trials of A versus nothing (i.e., surgery versus no surgery) and B versus nothing (i.e., PRP versus no PRP) would require 400 in each trial, assuming use of the same sample size estimate parameters. However, if interaction is observed in a factorial trial, then to estimate the effect of A, you need to compare A only (A0) versus neither (00); to estimate the effect of B, you need to compare B only (B0) versus neither (00) (i.e., “inside the table” analysis). Each of these comparisons involves only 200 patients (100 versus 100) and therefore is likely to be underpowered, and also involves the “neither” group twice, which raises the issue of correcting for multiple comparisons¹³. The latter case demonstrates where the efficiency of a factorial design can be lost.

In a typical factorial approach, there is a higher risk of making a type I and/or a type II error¹². For example, if there are no differences attributable to either factor, the probability of falsely concluding that one or the other or both factors are effective is close to 2α (type I error)¹². In regards to power, if both factors in a 2 × 2 trial are effective, the probability of concluding that both are effective is close to 1 to 2β. Hence the power of jointly detecting the efficacy of two interventions will be low¹². It has been suggested that to address these limitations, the sample size should be increased to have power 1 to β/2 for α/2 significance level tests¹². However, the monetary costs of the trial, feasibility of recruitment, costs of a type I error (monetary cost of the intervention or harm), the potential “opportunity costs” of a type II error (missed opportunity of using an effective intervention), and the potential costs of missing important antagonistic and negative interactions influence this choice¹.

In the FLOW study², the authors state that they will be able to efficiently and simultaneously investigate their two interventions (irrigating pressure and irrigating solution) by including all participants in both analyses.

Advantages and Disadvantages

Advantages

The advantages of a factorial design over a standard parallel group design include the ability to efficiently investigate a minimum of two interventions and the reduction in the total number of patients needed to assess multiple interventions aimed at achieving the same outcome¹. Certainly, the cost-effectiveness of such a design is an improvement over conducting multiple parallel group trials⁷. The ability to detect interactions is another proposed advantage of the factorial design over the parallel group design.

Disadvantages

Disadvantages include the inability to evaluate mutually exclusive interventions, potential for a higher type I error as a result of multiple testing, and, when all of the interventions are effective, there is an increase in the type II error in identifying that these treatments are indeed effective¹. Additionally, the decreased trial size in comparison to the parallel group design is only applicable when there are no interactions¹. Finally, most trials will be likely underpowered to detect significant interactions.

Summary

Since a certain amount of expertise is required to design, conduct, implement, and analyze factorial trials, it is recommended that a team of experts and statisticians familiar with factorial methodology in trials be consulted prior to initiating such factorial studies.

This review has highlighted the efficiency of the factorial design. Future research in orthopaedic surgery should attempt to take advantage of the cost-effectiveness and time-saving benefits that can be offered by such a design option. FLOW is an example of an orthopaedic study that has attempted to capitalize on the efficiency that is offered by factorial trials.

References

Gurusamy KS; Gluud C; Nikolova D; Davidson BR. Design of surgical randomized controlled trials involving multiple interventions. J Surg Res. 2011 Jan;165( 1):118-27. Epub 2009 Oct 30.[CrossRef][PubMed]

Flow Investigators. Fluid lavage of open wounds (FLOW): design and rationale for a large, multicenter collaborative 2 × 3 factorial trial of irrigating pressures and solutions in patients with open fractures. BMC Musculoskelet Disord. 2010 May 6;11:85.[CrossRef][PubMed]

Friedman LM; Furberg CD; DeMets DL. Fundamentals of clinical trials. New York: Springer New York; 2010.

Machin D; Fayers PM. Randomized clinical trials. West Sussex: Wiley-Blackwell; 2010.

McAlister FA; Straus SE; Sackett DL; Altman DG. Analysis and reporting of factorial trials: a systematic review. JAMA. 2003 May 21;289( 19):2545-53.[CrossRef][PubMed]

Montgomery AA; Peters TJ; Little P. Design, analysis and presentation of factorial randomised controlled trials. BMC Med Res Methodol. 2003 Nov 24;3:26.[CrossRef][PubMed]

Torgerson DJ; Torgerson CJ. Designing randomised trials in health, education and the social sciences. Hampshire: Palgrave Macmillan; 2008.

Fisher RA. The design of experiments. 9th ed.New York: Macmillan; 1971.

Wikipedia, The Free Encyclopedia. Factorial experiment. 2011 May 20. http://en.wikipedia.org/wiki/Factorial_experiment. Accessed 2011 May 29.

Little P; Roberts L; Blowers H; Garwood J; Cantrell T; Langridge J; Chapman J. Should we give detailed advice and information booklets to patients with back pain? A randomized controlled factorial trial of a self-management booklet and doctor advice to take exercise for back pain. Spine (Phila Pa 1976). 2001 Oct 1;26( 19):2065-72.[CrossRef][PubMed]

[No authors listed] Invasive compared with non-invasive treatment in unstable coronary-artery disease: FRISC II prospective randomised multicentre study. FRagmin and Fast Revascularisation during InStability in Coronary artery disease investigators. Lancet. 1999 Aug 28;354( 9180):708-15.[CrossRef][PubMed]

Green S; Liu PY; O’Sullivan J. Factorial design considerations. J Clin Oncol. 2002 Aug 15;20( 16):3424-30.[CrossRef][PubMed]

Montgomery AA. E-mail correspondence, June 16, 2011.

Topics

clinical trials ; research design ; factorial design

Username

Password

Access for Patients

Forgot Password?

Have a subscription to the print edition?

Not a Subscriber?

Buy this Article

Get online access for 30 days for $35

New to JBJS?

Subscribe to the online edition for 30 days for $75

Register for a FREE limited account to get full access to all CME activities, to comment on public articles, or to sign up for alerts.

Have a subscription to the print edition?

Current subscribers to The Journal of Bone & Joint Surgery in either the print or quarterly DVD formats receive free online access to JBJS.org.

Activate your online account now

Forgot your password?

Enter your username and email address. We'll send you a reminder to the email address on record.

Username:*

Email Address:*

Forgot your username or need assistance? Please contact customer service at subs@jbjs.org. If your access is provided
by your institution, please contact you librarian or administrator for username and password information. Institutional
administrators, to reset your institution's master username or password, please contact subs@jbjs.org

References

Accreditation Statement

These activities have been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Academy of Orthopaedic Surgeons and The Journal of Bone and Joint Surgery, Inc. The American Academy of Orthopaedic Surgeons is accredited by the ACCME to provide continuing medical education for physicians.

CME Activities Associated with This Article

Submit a Comment

Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discretion of JBJS editorial staff.

* = Required Field

Comment Author(s) * (if multiple authors, separate names by comma)

Example: John Doe

Affiliation & Institution*

Disclosure

Comment Title*

Comment*

Cancel

Print
PDF
Email

Recipient(s) will receive an email with a link (good for 5 days) to 'Efficient Designs: Factorial Randomized Trials'.
Your Name:*
Example: John Doe

Email Address:* CC Me:
Enter your valid email address. Example: jdoe@example.com

Recipient's Email Address:*

Separate multiple email address with semi-colons (up to 5).

Subject:* 's The Journal of Bone & Joint Surgery: 'Efficient Designs: Factorial Randomized Trials'
Subject for your email.

Message:

(Optional, message will truncate at 1000 characters)

Processing your request... Please Wait...

Copyright © in the material you requested is held by The Journal of Bone & Joint Surgery, Inc. (unless otherwise noted). This email ability is provided as a courtesy, and by using it you agree that that you are requesting the material solely for personal, non-commercial use, and that it is subject to Journal of Bone & Joint Surgery, Inc.'s Terms of Use. The information provided in order to email this topic will not be used to send unsolicited email, nor will it be furnished to third parties. Please refer to The Journal of Bone & Joint Surgery Inc.'s Privacy Policy for further information.

Copyright © The Journal of Bone & Joint Surgery Inc. All rights reserved.
Share
Get Citation

Daniel B. Whelan, Katie Dainty, Jaskarndip Chahal; Efficient Designs: Factorial Randomized Trials. The Journal of Bone & Joint Surgery. 2012 Jul;94(Suppl 1(E)):34-38.

Download citation file:

RIS (Zotero)

EndNote

BibTex

Medlars

ProCite

RefWorks

Reference Manager

Copyright © The Journal of Bone and Joint Surgery, Inc.
Rights & Permissions
Article Alerts

Processing your request... Please Wait.
Submit a Comment

The Journal of Bone & Joint Surgery

Randomized Trials in Orthopaedic Surgery: One Step Closer

J Bone Joint Surg Am, 2012 Jul 18;94(Suppl 1(E)):1-1
doi: 10.2106/JBJS.L.00311

Overcoming the Funding Challenge: The Cost of Randomized Controlled Trials in the Next Decade

J Bone Joint Surg Am, 2012 Jul 18;94(Suppl 1(E)):101-106
doi: 10.2106/JBJS.L.00193

Participating in Multicenter Randomized Controlled Trials: What's the Relative Value?

J Bone Joint Surg Am, 2012 Jul 18;94(Suppl 1(E)):107-111
doi: 10.2106/JBJS.L.00299

Lessons Learned from Benchmark Orthopaedic Trials

J Bone Joint Surg Am, 2012 Jul 18;94(Suppl 1(E)):11-14
doi: 10.2106/JBJS.K.01622

Observational Studies in the Era of Randomized Trials: Finding the Balance

J Bone Joint Surg Am, 2012 Jul 18;94(Suppl 1(E)):112-117
doi: 10.2106/JBJS.L.00242

JBJS Case Connector