Abstract
Background:
Traditionally, physicians have identified which outcome factors are important to measure in order to determine the success or failure of treatment without any input from patients. The purpose of the present study was to ascertain the five outcome factors that are most important to the patient and the impact that age and sex have on these factors. These five most important patient-derived outcome factors were then compared with factors within two of the most commonly used outcome instruments for the foot and ankle.
Methods:
Informant interviews, pre-testing, consistency analysis, and pilot testing led to the construction of a twenty-item survey of outcome factors that patients identified as being important in the treatment of their foot or ankle problem. Subjects selected the top five factors and rank ordered them from 1 to 5 (with 1 representing extreme importance and 5 representing least importance). One thousand computer simulations identified the top five factors, and these were subsequently stratified for sex and age. Wilcoxon rank-sum and Benjamini-Hochberg tests were used to compare the data between groups.
Results:
The survey was completed by 783 subjects. The five most important factors were limited walking (p < 0.05), activity-related pain (p < 0.05), constant pain (p < 0.05), difficulty with prolonged standing (p = 0.754), and inability to do one’s job or housework (p = 0.995). Shoe-related issues and foot and ankle weakness were significantly different between the sexes. Constant pain, inability to play sports, inability to participate in a job or housework, and recurrent foot or ankle skin sores or infections were significantly different between age groups. Between 38% and 50% of the outcome points found on two commonly used foot and ankle instruments included factors not of primary importance to the patient.
Conclusions:
There are sex and age-related differences regarding outcome factors following the treatment of disorders affecting the foot and ankle. As many as 50% of the factors in currently used foot and ankle outcome instruments are not of primary importance to patients.
The term clinical outcome began as a physician catch phrase in the 1990s and is commonly used by care providers, insurers, and patients today. Clinicians recognize the need to standardize the evaluation and reporting of the results of patient treatment to allow for effective interpretation and clinical application of scientific knowledge. Choosing the specific outcome instrument designed to meet these goals may be challenging.
Evaluation scales traditionally have included data thought to be clinically important to the physician. These measurements for the foot and ankle have focused on range of motion, pain, function, and radiographic criteria1-13. These data may be given either a numeric value or a categorical score, both of which suggest an excellent, good, fair, or poor result. These scales, although published and able to provide some insight into the patient response to treatment, do not allow a comparison of results due to the lack of standard measurement and reporting techniques. The majority of these scales have not undergone rigorous scientific evaluation to determine their reliability, validity, or responsiveness to change. Specifically, radiographic measurements of the foot that were developed in the past have poor interobserver reliability, demonstrating limited usefulness in outcome scales of the foot9,12,14.
Similarly, the reliability and validity of range-of-motion data involving the foot have been questioned15-17. Large arcs of joint motion may not correlate with improved function or decreased pain. This unsubstantiated link of range of motion with improved outcome highlights the potentially limited usefulness of absolute range of motion values in clinical rating scales. A recent study indicated that a high percentage (25% to 33%) of patients remain dissatisfied at the time of follow-up after the surgical treatment of foot problems14. This patient dissatisfaction occurs even when there is improvement in the physician-derived outcome measures of radiographic criteria and pain reduction, suggesting that patients may have different expectations and that other factors are more important to them. In the study by Wright et al., patients were asked to rate the importance of their complaints before undergoing total hip replacement and a patient-derived outcome score was calculated on the basis of the responses18. With this approach, the factors that were important to the patient were emphasized.
The purpose of the present study was to identify the factors that patients find important with regard to their foot problem and to compare the five most important factors with the physician-derived items currently used in foot outcome scales. The influence of patient age and sex on outcome factors was also examined.
Survey Instrument Construction
Ten consecutive patients (seven women and three men) with an average age of fifty-three years (range, twenty-four to eighty-seven years) were recruited from an outpatient orthopaedic foot and ankle practice and were interviewed individually with use of standard survey research open-ended questioning. The patients were asked to comment on “What problems would you like improved or eliminated with your foot or ankle treatment?” A complete list of responses was recorded. A total of seventy items were reported by the ten interviewees (range, four to twelve items per interviewee). In addition, subsequent pilot testing was performed on six additional patients through an “other” write-in option and no additional items were generated. The responses were categorized into the domains of activity, pain, orthotic/footwear issues, medication use, and activities of daily living (housework, driving, a job, and independence). Twenty items that were repeatedly reported by 70% of the patients were chosen for inclusion in the survey; these items included activity-related pain; constant pain; night pain; limp; stiffness; having to take a pill because of pain in the foot or ankle; having to use walking aids; crooked foot, ankle, or toes; fear of falling due to the foot or ankle; loss of independence; difficulty fitting into shoes; inability to play sports; inability to do one’s job or housework; difficulty with prolonged standing; need for bracewear or orthotics; limitations in walking; recurrent foot or ankle skin sores or infections; pain with driving; foot or ankle weakness; and foot or ankle numbness. An instruction and demographic information page was constructed for the survey, and pre-test analysis was performed to improve the wording and clarity of the instrument.
Additionally, a pilot test was offered to seventy-five patients with foot and ankle problems from three outpatient clinics at the University of Rochester Medical Center over a two-week period and sixty patients chose to complete the survey (response rate, 80%). The patients were asked to select the five most important items (factors) from the twenty-item list and to rank them from 1 (extremely important) to 5 (least important) as they related to the foot and/or ankle problem. There was a blank line to record any additional items. No additional items were suggested. The internal consistency was acceptable (r = 0.7). A power analysis was performed to determine the number of subjects required to assess if there were sex or age-specific differences in the patient-derived five most important factors of treatment outcome. With the anticipation of an 80% response rate, a minimum sample size of 475 subjects was determined a priori to provide 80% power to assess statistical differences at the 0.05 level.
Study Group Survey
Initial approval by the Research Subjects Review Board at the University of Rochester School of Medicine and Dentistry was obtained. During a three-month period (April 1 to June 30, 2008), patients with foot and ankle problems from three outpatient clinics were invited by the attending surgeon to complete the survey instrument in the examination room. Patients were excluded if they were younger than eighteen years of age, unwilling to participate, or unable to read, write, or understand the survey or the English language. The collected information included detailed demographic data including age, ethnicity, medical problems, and the survey item instrument. The data were recorded by the patient on a scanning form that allowed incorporation into an Excel spreadsheet for data analysis.
Data Collection and Statistical Analysis
Descriptive statistics were obtained from the demographic and ethnicity data. The five most important survey items were identified by the patients in order of most important to least important (1 to 5). The remaining fifteen items were given a score of 14, denoting not important, for statistical analysis. These scores are denoted by sik, where i = 1, 2, …, n are the indices of patients and k = 1, 2, … , K are the indices of the survey items. In this study, K = 20. For each survey item, we computed a summary score, 
. The top five items with the smallest summary scores were selected as the most important factors for the group. The following null hypothesis was tested for each one of the top five items: H0 : μk = μj, for all k, j = 1, 2, … , K. All survey items are assumed to have the same distribution of scores under this null hypothesis.
The statistical methodology is outlined below:
1. Compute Sk, the summary scores, without permutation. Sort these scores in ascending order and record the top five items as k1, k2, …, ks.
2. For each patient, randomly assign five items from K = 20 survey items. This amounts to permuting the index of top survey items from the pool of all items.
3. Compute the summary scores based on the permuted top items, and repeat this process for 1000 permutations. Denote them by 
, where p = 1, 2, … , 1000 are the indices of permutations.
4. For each permutation, sort from the smallest to the largest. These order statistics are denoted by 
, for k = 1, 2, …, K.
5. Assign permutation p values for the original selected top items by comparing their summary score with their permutation counterparts
With the five most important items identified in the group, descriptive statistics were used to define the percent agreement of the patient-derived five most important factors with two commonly used physician-derived outcome tools from the published literature. The two scales used in the present study were the Hallux Metatarsophalangeal-Interphalangeal scale and the Foot and Ankle Index-Revised scale (see Appendix). The American Orthopaedic Foot & Ankle Society (AOFAS) clinical rating system for the foot and ankle19 is an anatomic clinical outcome scale that has come into more widespread use in the past decade. It consists of four 100-point anatomic scales corresponding with the ankle-hindfoot, midfoot, hallux metatarsophalangeal-interphalangeal, and lesser metatarsophalangeal-interphalangeal joints. The Revised Foot Function Index is a self-administered questionnaire consisting of thirty-four items divided into five categories: pain, stiffness, difficulty, activity limitation, and social issues20. The data were stratified by age (less than fifty-five years and fifty-five years or more) to assess the impact of age on factor importance. An age of less than fifty-five years was selected as the cutoff for the younger age group because >50% of the United States workforce is still working at that age. At sixty years of age, only 43% of the workforce is working21.
For each factor, the Wilcoxon rank-sum test was applied to determine if there was a significant difference in the summary score for the two age groups. The Benjamini-Hochberg multiple-testing procedure was used to control the false discovery rate at the α = 0.05 level. The same analysis was also applied to determine the association between sex (male or female) and the summary scores.
Source of Funding
The role of the source of funding for this project was an AAOS/OREF health professions scholarship, funding supporting Masters of Public Health (MPH) education. The funds were used for tuition, books, and salary support.
During the three-month period, 821 subjects were seen. Thirty-eight subjects chose not to complete the study or were ineligible to participate, and 783 eligible subjects completed the survey. The means, ranges, and percentages for age, sex, and ethnicity are listed in Table I.
Table II provides a list of the twenty items, the ranking of each item, and the percentages of patients in each group who ranked each item as one of the top five most important factors. P values for the comparisons between the subgroups were reported. The top five factors important to the subject group were (1) limitations in walking (p < 0.05), (2) activity-related pain (p < 0.05), (3) constant pain (p < 0.05), (4) difficulty with prolonged standing (p = 0.754), and (5) inability to do a job or housework (p = 0.995).
When the data were stratified for sex, the five most important factors important to males were (1) limitations in walking, (2) activity-related pain, (3) constant pain, (4) inability to do a job or housework, and (5) difficulty with prolonged standing. The five most important factors for females were (1) limitations in walking, (2) activity-related pain, (3) constant pain, (4) difficulty fitting into shoes, and (5) difficulty with prolonged standing. Of the twenty factors that were ranked, difficulty fitting into shoes (p < 0.05) and foot and ankle weakness (p < 0.05) were significantly different between the sexes. Women ranked fitting into shoes as the fourth most important factor related to the foot, whereas men ranked this item as much less important.
When the data were stratified for age (less than fifty-five years or fifty-five years or more), the top five factors for the younger group were (1) activity-related pain, (2) limitations in walking, (3) constant pain, (4) inability to do a job or housework, and (5) inability to play sports. The top five factors for the older group were (1) limitations in walking, (2) activity-related pain, (3) constant pain, (4) difficulty fitting into shoes, and (5) difficulty with prolonged standing. Of the twenty factors that were ranked, constant pain (p < 0.05), inability to play sports (p < 0.05), inability to do a job or housework (p < 0.05), and recurrent foot or ankle skin sores or infections (p < 0.05) were significantly different between the age groups. Although constant pain was found to be in the top five factors for both the younger group and the older group, a significantly higher percentage of younger subjects than older subjects ranked this factor as important (25.8% compared with 16.6%; p < 0.05). The younger subjects ranked inability to play sports as the fifth most important factor, whereas the older group ranked it as less important. Similarly, the inability to do a job or housework was ranked as the fourth most important factor by the younger group and was ranked lower by the older group. Although the factor of recurrent foot or ankle skin sores or infections was significantly different between the age groups, with the older group ranking it very unimportant and the younger group ranking it the least important (p < 0.05), the percentage of subjects in each group who listed this factor as one of the top five important factors was very low (1.3% in the younger group, compared with 4.4% in the older group).
The five most important factors as defined by the subject group were compared with the two most commonly used physician-derived outcome scales19,20. Although each scale has a different format that does not allow direct comparison of numerical values allocated for each factor, the pain-related factors (activity-related pain and constant pain) and limitation in walking are well represented in both scales. The fourth most important factor in the overall group (difficulty with prolonged standing), representing a large number of subjects in the current survey (16.6%), is not included in either the AOFAS clinical rating scale or the Revised Foot Function Index.
On the 100-point AOFAS clinical rating scale for the hallux metatarsophalangeal-interphalangeal joints, 50 points are assigned to factors that are not of primary importance to the patient based on this overall ranking (footwear requirements, motion, stability, callus, alignment). Of the five components of the Revised Foot Function Index (pain, stiffness, difficulty, activity limitations, and social issues), two (stiffness and social issues), which represent thirteen of thirty-four total questions (38% of nonweighted points), were not significantly important to the patients in our study.
Clinical outcome scales may be divided into three types: disease-specific health-status measures22,23, generic health-status measures24, and anatomic scales19,20,25. A disease-specific instrument is designed to assess specific diagnostic groups or patient populations with the goal of measuring clinically important changes. Generic health-status measures are those that allow broad application across the severity of disease, different medical treatments, or health interventions and across demographic and cultural subgroups. These measures are designed to summarize a spectrum of health or quality-of-life issues that apply to many different impairments, illnesses, patients, and populations. Anatomic scales are specific for a particular body segment, such as the foot and/or ankle, are easy to use, and are frequently incorporated into clinical studies.
The AOFAS clinical rating system for the foot and ankle19 is an anatomic clinical outcome scale that has come into more widespread use in the past decade. It consists of four 100-point anatomic scales pertaining to the ankle-hindfoot, midfoot, hallux metatarsophalangeal-interphalangeal, and lesser metatarsophalangeal-interphalangeal joints (see Appendix). Three main categories (pain, function, and alignment) are included within each scale. Each scale is clinician-administered and includes subjective and objective criteria. The numerical weighting of data within each category is different for each anatomic scale. Patients may be asked a total of three to five questions, and the clinician completes the remaining data. The items include joint motion, gait abnormalities, stability, alignment and callus assessment, with higher scores corresponding with better outcomes. To our knowledge, no content or construct validity studies have been performed and no guidelines have been suggested for the interpretation of the data. Despite these limitations, the AOFAS scale has been extensively utilized and reported in the literature and may provide some general information on trends toward improvement or worsening of foot or ankle conditions.
Another commonly used anatomic foot scale is the Revised Foot Function Index20 (see Appendix). The Revised Foot Function Index is a self-administered questionnaire consisting of thirty-four items divided into five categories: pain, stiffness, difficulty, activity limitation, and social issues. All items are rated with use of a 6-point response scale, with higher scores corresponding with worse outcomes. Psychometric testing has been performed to validate this outcomes questionnaire20.
Although both clinical rating instruments are commonly used, physicians derived the items believed to be important in these scales without any apparent patient input into their clinical assessment19,26. The impact or relevance of patient input into these outcome scales is not known. In addition, the effect of patient sex or age on the identification of factors deemed important by the patient for their foot problem has not been studied.
In the present study, patients were asked what issues they would like to see resolved or improved as a result of foot or ankle treatment. The initial survey instrument was designed with the focus group being the patient and not the foot and ankle care provider. Pilot testing was performed with patients, and the survey had good internal consistency. The survey was distributed to a broad group of 783 subjects with foot and/or ankle complaints. The ethnicity of the group is a good representation of the population of upstate New York. The percentages of male and female respondents were similar to those in other studies, with women being 25% to 33% more commonly affected by foot and ankle problems than males. The top five factors important to patients included limitations in walking, constant pain, activity-related pain, difficulty with prolonged standing, and inability to do a job or housework. Of these top five items, the first three were found to be significant. In addition, these three items also represented the top three most important items for all ages and both sexes.
The factors that were statistically different for younger patients (patients less than fifty-five years of age) were related to the ability to play sports and to perform work responsibilities. This finding may be due to the fact that many patients in this age group participate in sports that involve running and jumping and are currently employed in the workforce, whereas patients in the older group are closer to the average retirement age for the United States population (sixty-five years).
The differences that were found between the sexes were also interesting. The factors of difficulty fitting into shoes and foot and/or ankle weakness were significantly different between men and women. Women thought that fitting into shoes was a very important issue, and men thought that it was in the lowest quartile of importance. This item may prove to be unnecessary to include in a scale given to men. The problem of foot and/or ankle weakness is more important to men (ranked seventh) than to women. Similarly, this weakness may be a sex-specific factor for a male-based scale and not important for a female-based scale.
The strengths of the present study are the survey instrument design, the internal consistency analysis, and the computerized simulation modeling providing quantitative analysis of rank-ordered values for a large number of subjects.
The potential limitations of a study of this type include the diversity of foot and ankle problems among patients presenting to a tertiary-care academic facility. There was no effort to stratify the problems to the forefoot, hindfoot, midfoot, or ankle as is done with the AOFAS clinical rating scale. The wording of the patient-derived factors was provided by the patients during the survey construction, and no changes were made to further define these factors. Patient interpretation of these factors could be variable. Comparison of these patient-worded factors did not exactly mimic the wording in the AOFAS clinical rating scale or the Revised Foot Function Index, and therefore comparisons were made on the basis of the wording intent; this may be considered a limitation. This study had a large proportion of white subjects, and therefore the generalizability to other ethnicities would require additional testing.
The present study examined factors that patients find important when assessing the outcome of foot care. There are sex and age-related differences that should be taken into account when assessing the outcome of operative or nonoperative treatment. We discovered that up to half of each survey instrument from two commonly used physician-derived foot and ankle rating scales may be composed of elements that are not of primary importance to patients19,20. Additional research and validation are needed to produce a clinically relevant outcome instrument that includes factors that are both important to the patient and useful for the clinician.
Tables showing the details of the Hallux Metatarsophalangeal-Interphalangeal Scale and the Revised Foot Function Index are available with the online version of this article as a data supplement at jbjs.org.
Note: The contents of this paper do not represent the views of the Department of Veterans Affairs or the United States Government.
Hildebrand
KA;
Buckley
RE;
Mohtadi
NG;
Faris
P. Functional outcome measures after displaced intra-articular calcaneal fractures. J Bone Joint Surg Br.
1996 Jan;78(
1):119-23.
Kitaoka
HB;
Franco
MG;
Weaver
AL;
Ilstrup
DM. Simple bunionectomy with medial capsulorrhaphy. Foot Ankle.
1991 Oct;12(
2):86-91.
Kitaoka
HB;
Holiday
AD
Jr. Metatarsal head resection for bunionette: long-term follow-up. Foot Ankle.
1991 Jun;11(
6):345-9.
Kofoed
H;
Sørensen
TS. Ankle arthroplasty for rheumatoid arthritis and osteoarthritis: prospective long-term study of cemented replacements. J Bone Joint Surg Br.
1998 Mar;80(
2):328-32.[CrossRef]
McGuire
MR;
Kyle
RF;
Gustilo
RB;
Premer
RF. Comparative analysis of ankle arthroplasty versus ankle arthrodesis. Clin Orthop Relat Res.
1988 Jan;(
226):174-81.
Merchant
TC;
Dietz
FR. Long-term follow-up after fractures of the tibial and fibular shafts. J Bone Joint Surg Am.
1989 Apr;71(
4):599-606.
Myerson
MS;
Fisher
RT;
Burgess
AR;
Kenzora
JE. Fracture dislocations of the tarsometatarsal joints: end results correlated with pathology and treatment. Foot Ankle.
1986 Apr;6(
5):225-42.
Parmar
HV;
Triffitt
PD;
Gregg
PJ. Intra-articular fractures of the calcaneum treated operatively or conservatively. A prospective study. J Bone Joint Surg Br.
1993 Nov;75(
6):932-7.
Perry
MD;
Mont
MA;
Einhorn
TA;
Waller
JD. The validity of measurements made on standard foot orthoroentgenograms. Foot Ankle.
1992 Nov-Dec;13(
9):502-7.
Platto
MJ;
O’Connell
PG;
Hicks
JE;
Gerber
LH. The relationship of pain and deformity of the rheumatoid foot to gait and an index of functional ambulation. J Rheumatol.
1991 Jan;18(
1):38-43.
Smith
RW;
Reynolds
JC;
Stewart
MJ. Hallux valgus assessment: report of research committee of American Orthopaedic Foot and Ankle Society. Foot Ankle.
1984 Sep-Oct;5(
2):92-103.
Steel
MW
3rd;
Johnson
KA;
DeWitz
MA;
Ilstrup
DM. Radiographic measurements of the normal adult foot. Foot Ankle.
1980 Nov;1(
3):151-8.
Zvijac
JE;
Janecki
CJ;
Freeling
RM. Distal oblique osteotomy for tailor’s bunion. Foot Ankle.
1991 Dec;12(
3):171-5.
Saltzman
CL;
Brandser
EA;
Berbaum
KS;
DeGnore
L;
Holmes
JR;
Katcherian
DA;
Teasdall
RD;
Alexander
IJ. Reliability of standard foot radiographic measurements. Foot Ankle Int.
1994 Dec;15(
12):661-5.
Elveru
RA;
Rothstein
JM;
Lamb
RL. Goniometric reliability in a clinical setting. Subtalar and ankle joint measurements. Phys Ther.
1988 May;68(
5):672-7.
Smith-Oricchio
K;
Harris
BA. Interrater reliability of subtalar neutral, calcaneal inversion and eversion. J Orthop Sports Phys Ther.
1990;12(
1):10-5.[PubMed]
Nawoczenski
DA;
Baumhauer
JF;
Umberger
BR. Relationship between clinical measurements and motion of the first metatarsophalangeal joint during gait. J Bone Joint Surg Am.
1999 Mar;81(
3):370-6.
Wright
JG;
Rudicel
S;
Feinstein
AR. Ask patients what they want. Evaluation of individual complaints before total hip replacement. J Bone Joint Surg Br.
1994 Mar;76(
2):229-34.
Kitaoka
HB;
Alexander
IJ;
Adelaar
RS;
Nunley
JA;
Myerson
MS;
Sanders
M. Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int.
1994 Jul;15(
7):349-53.
Budiman-Mak
E;
Conrad
K;
Stuck
R;
Matters
M. Theoretical model and Rasch analysis to develop a revised Foot Function Index. Foot Ankle Int.
2006 Jul;27(
7):519-27.
Retirement in specific countries. .
Bellamy
N;
Buchanan
WW;
Goldsmith
CH;
Campbell
J;
Stitt
LW. Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol.
1988 Dec;15(
12):1833-40.
Martin
DP;
Engelberg
R;
Agel
J;
Swiontkowski
MF. Comparison of the Musculoskeletal Function Assessment questionnaire with the Short Form-36, the Western Ontario and McMaster Universities Osteoarthritis Index, and the Sickness Impact Profile health-status measures. J Bone Joint Surg Am.
1997 Sep;79(
9):1323-35.
McHorney
CA;
Ware
JE
Jr;
Rogers
W;
Raczek
AE;
Lu
JF. The validity and relative precision of MOS short- and long-form health status scales and Dartmouth COOP charts. Results from the Medical Outcomes Study. Med Care.
1992 May;30(
5 Suppl):MS253-65.[CrossRef]
American Academy of Orthopaedic Surgeons. .
Walmsley
S;
Williams
AE;
Ravey
M;
Graham
A. The rheumatoid foot: a systematic literature review of patient-reported outcome measures. J Foot Ankle Res.
2010 Jul 9;3:12.[CrossRef]