The Journal of Bone & Joint Surgery

The Journal of Bone & Joint Surgery, Volume 91, Issue Supplement 3

Design, Conduct, and Interpretation of Nonrandomized Orthopaedic Studies--A Practical Approach | May 01, 2009

Survey Design in Orthopaedic Surgery: Getting Surgeons to Respond

Sheila Sprague, MSc¹; Laura Quigley, BSc¹; Mohit Bhandari, MD, MSc, FRCSC¹

¹ Division of Orthopaedic Surgery, Department of Surgery (M.B.) and Department of Clinical Epidemiology and Biostatistics (S.S., L.Q., M.B.), McMaster University, 293 Wellington Street North, Suite 110, Hamilton, ON L8L 2X2, Canada. E-mail address for M. Bhandari: bhandam@mcmaster.ca

View Disclosures and Other Information

Disclosure: The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity.

The Journal of Bone and Joint Surgery, Inc.

J Bone Joint Surg Am, 2009 May 01;91(Supplement 3):27-34. doi: 10.2106/JBJS.H.01574

5 Recommendations (Recommend) | 3 Comments | Saved by 3 Users Save Case

Article

Comments (0)

text A A A

Abstract

We provide an overview of survey design and implementation strategies in orthopaedic surgery. Health-care surveys are vital for obtaining information on the beliefs, patterns of practice, attitudes, and behaviors of orthopaedic surgeons. It is important to obtain a high response rate from administered surveys to reduce bias due to nonresponse. Researchers should follow the guidelines provided by this review to increase the response rate of orthopaedic surgeons to surveys. When designing these surveys, the researcher must consider length, format, and aesthetics. In addition, the types of questions that are included, the wording of these questions, and the order in which the questions are presented within the survey need to be carefully considered. Surveys can be administered by telephone, mail, facsimile (fax), and electronically by e-mail or Internet. The use of a mixed-mode method is recommended to improve the response rate. To increase the response rate to surveys that are directed at health professionals, a number of strategies have been suggested, including using cover letters, personalizing the cover letter and survey package, pretesting the cover letter and survey, contacting the surgeons prior to administration of the survey, contacting the surgeons multiple times, using stamped return envelopes in mail surveys, using appropriate survey packaging styles, providing incentives, and ensuring that the orthopaedic surgeon recognizes the sender of the survey. The costs associated with each administration method are briefly discussed, and ethical considerations are reviewed.

Figures in this Article

Introduction

Health-care surveys are an important tool for gaining information about the beliefs, practice patterns, knowledge, attributes, behaviors, and attitudes of orthopaedic surgeons^1-3. The results of large surveys often shape the research needs for the field, and surveys can further shed light on the adoption of current evidence to surgical practice.

Survey validity is threatened by low response rates^1,4-9. Busy workloads and low priority have led to low response rates^6,9-13 that have been as low as 15% among surgeons¹³. Bias due to nonresponse appears when survey analysis is limited to the available data, as responders to questionnaires may be different from nonresponders, creating a nonrepresentative sample^3,14. Low response rates can increase the nonresponder bias in a survey and jeopardize its ultimate credibility to guide decision-making in orthopaedic surgery¹⁵. Biases are typically limited by achieving response rates of 70% or higher³. Not only does an increased response rate correspond to increased validity of the survey, but it can also lead to reduced costs because of the decreased need to resend surveys^15,16.

Survey Design

There are a variety of implications to consider in designing a survey directed at orthopaedic surgeons. Jones et al. proposed twelve principles for conducting a survey, and these principles are aimed at increasing the response rate (Table I)¹⁷. Aspects of survey design that are vital include defining the purpose of the survey; developing questions that will provide insight to this purpose; designing questions that are simple, unambiguous, and brief; and identifying a target population¹⁷. Pretesting and piloting the survey with a representative sample will help to ensure that it is understood by those who will complete it¹⁷.

View Larger

TABLE I Twelve Principles for Conducting a Survey¹⁷*

1. Clearly define the purpose and objectives of the survey

2. Give the survey an appropriate, accurate title

3. Make the survey as brief and simple as possible

4. Keep each question short, simple, unambiguous, and unidimensional. The questions should be designed to specifically answer the study objectives

5. Avoid questions and data-collection techniques that influence the answers

6. Decide how the data will be compiled and analyzed before conducting the survey

7. Identify and target a representative and appropriately sized sample of the overall target population. Quantify the response rate to the survey and assess the characteristics of respondents and nonrespondents

8. Pretest the survey, ideally with a representative sample of the focus population that is not going to be included in the sample to be used for the final survey

9. Revise the questionnaire following the results of the pretest, and pretest again if necessary

10. Distribute the questionnaire to the broader sample

11. Allow space on the survey for voluntary additional comments

12. Always thank the respondents

Reproduced, with modification, from: Jones D, Story D, Clavisi O, Jones R, Peyton P. An introductory guide to survey research in anaesthesia. Anaesth Intensive Care. 2006;34:246. Reprinted with permission.

Add to My JBJS

TABLE I Twelve Principles for Conducting a Survey¹⁷*

1. Clearly define the purpose and objectives of the survey

2. Give the survey an appropriate, accurate title

3. Make the survey as brief and simple as possible

4. Keep each question short, simple, unambiguous, and unidimensional. The questions should be designed to specifically answer the study objectives

5. Avoid questions and data-collection techniques that influence the answers

6. Decide how the data will be compiled and analyzed before conducting the survey

8. Pretest the survey, ideally with a representative sample of the focus population that is not going to be included in the sample to be used for the final survey

9. Revise the questionnaire following the results of the pretest, and pretest again if necessary

10. Distribute the questionnaire to the broader sample

11. Allow space on the survey for voluntary additional comments

12. Always thank the respondents

The design of the survey involves decisions about overall appearance, length, and the sequence of the questions. These elements influence response rates and the validity and quality of received responses³. The current standard for survey design, as advocated by Dillman, consists of a tailored design. A tailored design incorporates the development of survey procedures that create respondent trust and perceptions of increased rewards and reduced costs for being a respondent¹⁴. Table II provides a summary of the aspects of a survey that should be considered in designing a survey directed at orthopaedic surgeons.

View Larger

TABLE II Designing a Survey

	Survey Design
Item	Recommended	Not Recommended
Length of survey	8.5"× 14" (21.6 × 35.6 cm) piece of paper	8.5"× 11" (21.6 × 35.6 cm) piece of paper
	As short as possible while still obtaining the required information	Booklet format
Format	Single-sided printing	Double-sided printing
	Portrait layout	Landscape layout
	Precise instructions where needed	Unusual folds
	Navigational paths	Unusual-shaped pages
	Extra spaces between questions	All instructions at the beginning of the survey
	Consecutive numbering of questions
	Vertical listing of response options	Horizontal listing of response options
Aesthetics	Simple, natural graphical design	Detailed graphical design
	Colored paper
Type of questions	Simple questions	Open-ended questions
	Questions that surgeons will be able to recall	"Other" option
		Response ranking
		Neutral or midpoint answer choices
Question wording	Simple wording	Overly precise wording
	As few words as possible, while still creating complete sentences	Vague wording
	Exclusive response categories	Same categories for many question-and-answer choices
	Technically accurate wording	Respondent must make unnecessary calculations
		Double-barreled questions

Add to My JBJS

TABLE II Designing a Survey

	Survey Design
Item	Recommended	Not Recommended
Length of survey	8.5"× 14" (21.6 × 35.6 cm) piece of paper	8.5"× 11" (21.6 × 35.6 cm) piece of paper
	As short as possible while still obtaining the required information	Booklet format
Format	Single-sided printing	Double-sided printing
	Portrait layout	Landscape layout
	Precise instructions where needed	Unusual folds
	Navigational paths	Unusual-shaped pages
	Extra spaces between questions	All instructions at the beginning of the survey
	Consecutive numbering of questions
	Vertical listing of response options	Horizontal listing of response options
Aesthetics	Simple, natural graphical design	Detailed graphical design
	Colored paper
Type of questions	Simple questions	Open-ended questions
	Questions that surgeons will be able to recall	"Other" option
		Response ranking
		Neutral or midpoint answer choices
Question wording	Simple wording	Overly precise wording
	As few words as possible, while still creating complete sentences	Vague wording
	Exclusive response categories	Same categories for many question-and-answer choices
	Technically accurate wording	Respondent must make unnecessary calculations
		Double-barreled questions

Survey Length

The survey should be as short as possible to help increase the response rate and limit responder fatigue¹⁷. Questions of interest to the surveyor but not vital to answering the primary research question should be limited or not included¹⁷. Jepson et al. found, in a study on physicians, that questionnaire length above a threshold level of 1000 words was associated with lower response rates¹⁸. The threshold of 1000 words may not directly apply to surveys aimed at orthopaedic surgeons, but the finding of Jepson et al. does suggest that survey response rates are sensitive to relatively small changes in survey length¹⁸.

Survey Format

Having an appealing format in a survey is important in obtaining a high response rate among orthopaedic surgeons. There are certain formats of surveys that should be avoided, including: (1) printing on both sides of the sheets of paper and use of a staple to hold the pages together, (2) printing pages in a landscape layout rather than in a portrait layout, (3) incorporating unusual folds, and (4) using unusually shaped pages^12,14. Clear instructions should be provided throughout the survey, as opposed to only providing instruction at the beginning of the survey. A visual navigational guide should be created with use of symbols and arrows, which will assist the respondent in following the questionnaire in the correct order and which should help the respondent to skip ahead, depending on the response to a previous question, as appropriate¹⁴.

Dillman recommends visually emphasizing important words and phrases and numbering questions consecutively and simply¹⁴. Ensuring that adequate spaces are placed between questions, using dark print for questions and lighter print for answers, listing answer categories vertically instead of horizontally, avoiding double or triple-banking of answer choices, and using shorter lines to prevent words from being skipped are all recommended when formatting a survey to the needs and preferences of orthopaedic surgeons¹⁴.

Survey Aesthetics

The appearance of the questionnaire can be a contributing factor in an orthopaedic surgeon's decision to complete and return it. The questionnaire should be immediately distinguishable from all other questionnaires that a respondent may receive, and it should create a positive first impression¹⁴.

Types of Questions to Include

In deciding which questions to include in a survey, researchers should ensure that the question is necessary, decide whether several questions are required to adequately cover the subject, determine whether the respondent will have and will provide the desired information, and ensure that the question is valid^3,14. Keeping recall simple and related to recent events helps to produce high-quality survey data, and, by decreasing vagueness in question-and-answer categories, there will be a smaller potential for measurement error¹⁴. Some question types that should be avoided within the survey are open-ended questions, the option "other," a response (such as "unsure" or "unknown") at the scale midpoint, neutral responses, and asking respondents to rank responses⁸.

Wording of the Question

The proper phrasing of questions is vital to the validity of the measurements obtained from the survey³. A poor choice in wording can lead to questions that are excessively vague, excessively precise, misunderstood, confusing, objectionable, uninteresting, or irrelevant³. Dillman has crafted an approach to question-writing and suggests using simple rather than specialized words, choosing as few words as possible to pose the question, using complete sentences to ask questions, avoiding vague quantifiers when more precise estimates can be obtained (that is, do not use the same categories for many different kinds of questions), and avoiding specificity that exceeds the respondent's potential for having an accurate, ready-made answer¹⁴. He also recommends using equal numbers of positive and negative categories for scalar questions, distinguishing undecided answers from neutral responses by placing the option for a response of "undecided" at the end of the scale (if it is placed in the middle of the scale it is more likely to be chosen), avoiding bias from unequal comparisons, stating both sides of attitude scales in the question stem, and developing response categories that are mutually exclusive¹⁴. It is important to ensure that each question is technically accurate. Avoid asking respondents to say yes in order to mean no, avoid double-barreled questions, soften the impact of potentially objectionable questions, and avoid asking respondents to make unnecessary calculations¹⁴.

Order of Questions

The order of the questions is a particular aspect of the survey that influences the response rate as well as the validity and quality of the responses³. The questions should be ordered according to importance, content area, and the degree to which they might be objectionable¹⁴. The first question is the most important question of the survey, as it will influence readers with regard to whether they will complete the questionnaire or discard it¹⁴. The first question on the survey should clearly apply to everyone being surveyed; it should be easy to answer, and it should be interesting¹⁴. Mullner et al. found that questionnaires that began with questions about the most relevant topics achieved an overall response rate of 72%, whereas the questionnaires that began with questions about less salient and less relevant topics achieved an overall response rate of 67%¹⁹.

Respondents, who can be influenced by earlier questions and can detect if those questions convey a favorable or unfavorable impression, can become sensitized to the researcher's expectations³. Also, responses to some questions may influence a response to another question³. Personal and confidential questions should be placed at the end of the questionnaire^8,17. The filter technique is used when respondents are directed to omit particular questions based on answers to previous questions³. This technique is useful because it limits a respondent's frustration when he or she comes across a question that does not appear to apply, thus keeping the respondent from thinking that the entire survey is not applicable, which may lead to discarding of the questionnaire³.

Methods of Survey Administration

There are many methods of survey administration, including telephone, postal, fax, and electronic administration (Table III).

View Larger

TABLE III Advantages and Disadvantages of Various Methods of Survey Administration

Method of Survey Administration	Advantages	Disadvantages
Telephone	Increased accuracy of responses	Difficulty in accessing intended physician
	Few missed questions	Interviewer bias
		Interviewer distortion
Mail	Increased response rate	Incorrect mailing addresses
	Convenient	Ongoing costs
	Ease of surveying geographically dispersed populations	Data recording can be inaccurate
	Confidentiality and anonymity can be ensured	Longest delay in return time
	Increased validity of answers	Surgeons may not be able to explain responses as well compared with a telephone interview
Fax	Can record data via optical character recognition	Can be expensive
	Fast delivery and return	Highest number of incomplete questions
Internet and e-mail	Cost-effective	Decreased response rate
	Faster responses	Surgeon may be unfamiliar with Internet or e-mail
	Easy to send reminders	Easier ability to delete
	Reduced labor
	Decreased amount of questions that have been left unanswered	Out-of-date e-mailing lists
	Results can be quickly pooled and analyzed	Survey may be viewed as junk mail or computer virus

Add to My JBJS

TABLE III Advantages and Disadvantages of Various Methods of Survey Administration

Method of Survey Administration	Advantages	Disadvantages
Telephone	Increased accuracy of responses	Difficulty in accessing intended physician
	Few missed questions	Interviewer bias
		Interviewer distortion
Mail	Increased response rate	Incorrect mailing addresses
	Convenient	Ongoing costs
	Ease of surveying geographically dispersed populations	Data recording can be inaccurate
	Confidentiality and anonymity can be ensured	Longest delay in return time
	Increased validity of answers	Surgeons may not be able to explain responses as well compared with a telephone interview
Fax	Can record data via optical character recognition	Can be expensive
	Fast delivery and return	Highest number of incomplete questions
Internet and e-mail	Cost-effective	Decreased response rate
	Faster responses	Surgeon may be unfamiliar with Internet or e-mail
	Easy to send reminders	Easier ability to delete
	Reduced labor
	Decreased amount of questions that have been left unanswered	Out-of-date e-mailing lists
	Results can be quickly pooled and analyzed	Survey may be viewed as junk mail or computer virus

Telephone Surveys

Telephone surveys have the capability of increasing the accuracy of the response because orthopaedic surgeons can describe exactly what they mean by their response. In addition, the interviewer can ensure that all questions are answered. A major obstacle in conducting telephone surveys and achieving a high response rate is the ability to access the intended orthopaedic surgeon⁵. Surveys administered by telephone may also be prone to interviewer bias and distortion.

Postal Surveys

Traditionally, surveys have been administered by postal delivery. A postal survey consists of a cover letter and a survey that is mailed individually to each orthopaedic surgeon. The survey is completed by hand and then returned via postal mail. Postal surveys are popular because they ensure confidentiality, anonymity, and convenience for the respondent while minimizing social desirability bias and avoiding interviewer distortion^3,10. In addition, postal surveys have the advantage of being associated with an increased validity of responses because the respondents have the opportunity to consult other sources of information. Also, because they can be completed in private, postal surveys may be superior for sensitive issues³.

There are several difficulties and disadvantages to conducting postal surveys. There are ongoing costs for personal time and supplies, including postage, paper, and envelopes²⁰. Also, because data recording of postal surveys is completed by hand, it may not be as accurate as the data recording of an electronic survey, which is automatically entered into a database²⁰.

Facsimile (Fax) Surveys

A facsimile or fax survey consists of a cover letter and survey that is printed and then faxed to each orthopaedic surgeon. Respondents return the survey by either fax or postal mail, and the data may be recorded by hand or by optical character recognition by some fax machines²⁰. In comparison with postal surveys, the option of using optical character recognition on faxed surveys may increase the accuracy of the recorded data²⁰. The cost of a fax survey is similar to the cost of a postal survey.

Electronic Surveys

With the rapidly advancing technology that is available in the workplace, the use of e-mail and the Internet has become an increasingly popular option for conducting surveys among orthopaedic surgeons^13,21-27. Electronic surveys require computers with Internet access on the part of both the sender and the receiver²⁴. When Internet administration is chosen, the orthopaedic surgeon receives an e-mail (similar to the cover letter in a postal survey) that invites him or her to complete the survey and supplies a link to the survey's web site. The orthopaedic surgeon then logs on to the web site with use of a unique user name and password, accesses the survey, and completes it. The survey results are then downloaded by the surveyor. An e-mail administered survey is similar in that the surgeon receives an e-mail inviting him or her to complete the survey. The survey can be displayed as text within the e-mail or sent as an attachment to the e-mail²⁰. The surveyor should provide the respondent with the option to print an electronic survey and then submit it through postal mail or fax²⁴.

There are many advantages to electronic administration of surveys. Electronic surveys are technically easy to complete. In addition, they provide the potential to conduct large-scale surveys while eliminating the costs of stationery, postage, and administration; the cost of sending multiple e-mail invitations and reminders is negligible^21,24,25. Major benefits of Internet surveys also include the immediate availability of responses and the ability to download the data directly into a database for the purpose of analysis, thus reducing delay, manpower resources, and data-entry errors²⁶. In addition, electronic surveys are generally filled out and sent in immediately, whereas postal surveys may not be placed in the mail until a couple of days after the survey has been completed^13,20-25. Electronic surveys provide the possibility of controlling the order in which respondents see specific questions, preventing respondents from returning to change previous answers²⁴. E-mail surveys can be useful in their ability to limit the number of responses to one per question, and, in comparison with posted or fax surveys, they also appear to be associated with a decreased amount of unanswered questions²⁰. Also, e-mail may be a better mode for surveys with open-ended questions because responders seem to be willing to write longer responses by e-mail than by hand^21,22.

Disadvantages of electronic administration of surveys to orthopaedic surgeons include the possibility of the surgeon viewing the survey as junk mail, the surgeon's unfamiliarity with the Internet, and out-of-date e-mailing lists^20,25. The process of obtaining e-mail addresses through mailing address lists can be labor-intensive²³. The wide variety of Internet browsers and operating platforms may result in the survey being displayed differently among participants^13,24.

Which Mode Provides the Best Response Rate?

The convenience and low cost of the electronic methods of data collection are offset by the possibility of a lower response rate^13,20-24,26. Postal and fax surveys typically achieve a higher response rate than that achieved by telephone or electronic surveys^{5,13,20,22,23}. The use of mixed-mode designs can contribute to an increased response rate¹⁴. An example of the mixed-mode design would be distribution of the survey by a particular method for the first administration (for example, through e-mail or Internet mode) and then, for the second administration, sending the survey through a different method (for example, by postal mail).

Improving the Response Rate

A wide variety of strategies have been suggested to help improve the response rate for surveys that are administered to physicians and surgeons (Table IV).

View Larger

TABLE IV Methods of Increasing Response Rates

Method	Increases Response Rate	Mixed Results	Does Not Increase Response Rate
Cover letter	X
Personalization	X
Pretesting	X
Multiple contacts	X
Precontact		X
Incentives		X
Stamped return envelope	X
Paper quality			X
Survey packaging	X
Mailing date	X
Deadlines			X
Opinion leader			X
Knowing the surgeon	X

Add to My JBJS

TABLE IV Methods of Increasing Response Rates

Method	Increases Response Rate	Mixed Results	Does Not Increase Response Rate
Cover letter	X
Personalization	X
Pretesting	X
Multiple contacts	X
Precontact		X
Incentives		X
Stamped return envelope	X
Paper quality			X
Survey packaging	X
Mailing date	X
Deadlines			X
Opinion leader			X
Knowing the surgeon	X

Cover Letters

Cover letters are the first—and sometimes the only—opportunity to convince the reader to complete and return the survey³. They should accompany all surveys, regardless of the mode of administration. The cover letter should be limited to one page, and it should include the date, salutation, an introductory paragraph of what the survey is about, and the reason why completion of the survey is useful and important¹⁴. The cover letter should offer a guarantee of confidentiality and anonymity and state that participation is voluntary^3,14. The cover letter should convey details such as who should be contacted if questions arise and the deadline date for returning the questionnaire, and it should thank the responding orthopaedic surgeon for his or her time¹⁴.

Personalization

Correspondence can be personalized in a variety of ways, including the use of high-quality stationery, the use of real names instead of a preprinted salutation, and the use of real signatures^14,28. Maheux et al. found that a handwritten thank-you note placed on the bottom of a letter accompanying a copy of the questionnaire increased physicians' returns by 40.7%⁷. It was also found that personalizing the mail-out package, by individually typing the physician's title, name, and address on each envelope and then hand-stamping the envelope, increased the response rate by 53.1%⁷. Orthopaedic surgeons who received a personalized cover letter that emphasized the importance of the surgeon's contribution to the survey were more likely to respond to the initial mailing of postal mail surveys than surgeons who received a standard cover letter¹. These results, however, do not extend to electronic administrations of the same cover letters and surveys¹.

Pretesting

Pretesting can be crucial to the success of questionnaire delivery. The pretesting should duplicate all proposed methods of sampling, data collection, and administration of the survey³. The pretest can provide information on the effects of varying the locations to which the survey is being sent, and it can alert administrators to the need to debug administrative procedures. It can also aid in testing analytical methods, constructing dummy tables, and estimating costs, standards errors, and the required sample size³. Generally, the pretest should include a pilot study and a review by knowledgeable colleagues; after that, a few people who have not been involved at all with the questionnaire should complete it to see if any mistakes are still hidden¹⁴. The cover letter should be pretested to determine whether it conveys the intended message and activates the reader to participate in the study³.

Precontacts and Multiple Contacts

An additional technique that may lead to an increased response rate is the use of a so-called primer postcard, or, alternatively, a telephone call, to inform the intended surgeon of a forthcoming survey. One study revealed, however, that prenotification letters did not appear to be of any value in increasing the response rate to surveys⁹. Other studies have found that questionnaires that are administered with advance letters and postcards have a higher and faster response rate than do questionnaires that are administered without precontact^3,14. With regard to electronic surveys, since many may be considered "junk mail" and deleted, an initial contact is important to their success²⁶. A prenotification letter should be personalized, brief, positively worded, and aimed at building anticipation rather than providing the details or conditions for participation in the survey¹⁴.

Currently, the tailored design method as suggested by Dillman is the standard for conducting postal mail and Internet surveys (Table V)¹⁴. Each contact is slightly different from the others, thus conveying a renewal of effort on the part of the researcher¹⁴. Regardless of the method utilized for contacting nonresponders, research indicates that multiple contacts are the most effective way to increase the response rate⁸. Dillman also suggests implementation of the mixed-mode design if the initial contact was unsuccessful¹⁴.

View Larger

TABLE V The Dillman Approach¹⁴

Contact Number	Method of Contact
1	Prenotification letter via postal mail
2	Questionnaire mailing with cover letter
3	Replacement questionnaire sent to nonrespondents
4	Final contact via telephone or mail

Add to My JBJS

TABLE V The Dillman Approach¹⁴

Contact Number	Method of Contact
1	Prenotification letter via postal mail
2	Questionnaire mailing with cover letter
3	Replacement questionnaire sent to nonrespondents
4	Final contact via telephone or mail

Incentives

A variety of incentives, including money, lottery tickets, vouchers, pens, and other prizes, have been utilized in an attempt to increase the response rate of surveys^4,5,10,29-34. Investigators should carefully consider the nature of the incentive that is offered so that it is optimally appealing to the target group². Although some studies have found that incentives increase the response rate, the majority have found that the response rate is not increased. The additional costs of the incentives and the limited yield in response may indicate that incentives are not cost-effective.

Stamped Return Envelopes

Providing stamped return envelopes has been shown to improve the response to a survey, with minimal additional cost⁹. Stamped return envelopes should be individually stamped envelopes, not business reply envelopes, as a recent study found that physicians are more likely to return stamped envelopes than business reply envelopes²⁹.

Survey Packaging

The packaging of a survey may influence the response rate. Asch and Christakis found that the response rate among individuals receiving a Veterans Affairs envelope was 20% higher than the response rate among those receiving a university envelope¹⁶. Researchers should take the packaging of their survey, if administered by postal delivery, into consideration and should pilot several alternative mailing strategies, perhaps as part of the pretesting, before commencing a large mailing¹⁶.

Administration Date

When determining administration dates for the questionnaire, the characteristics of the orthopaedic-surgeon population should be considered. Dillman suggests that January through March is the best time to administer surveys¹⁴. October through December is a time period to be avoided due to the increase in mail activity during the holiday season^3,14.

Deadlines

The inclusion of deadlines for submission of completed surveys does not appear to increase or decrease ultimate response rates, but it has been found that deadlines do accelerate the return of questionnaires³. Providing a deadline also helps in the timing and planning of the second contacts of the nonresponders.

Opinion Leaders

Opinion leaders (influential individuals who are trusted in the community) have been shown to influence the practice of health professionals and patient outcomes⁴. Surprisingly, Bhandari et al. found that the response rate among surgeons who received a questionnaire along with an accompanying opinion-leader endorsement letter was lower than the response rate among surgeons who received a survey without an endorsement letter⁴.

Recognition of the Sender

A recent study found that those who received correspondence from a well-known investigator were 6% more likely to respond¹². Location is also an important determinant of response rate; Brehaut et al. found that physicians from the same Canadian province as the sender are marginally more likely than physicians from other Canadian provinces to respond to a survey¹². Response rates may be improved by combining the location of the surveyor and the recognition perceived by prospective respondents, thus ensuring that the survey comes from a well-known, local expert in the field¹².

Costs Associated with Surveys

Costs associated with surveys vary according to the method of survey administration and include the labor costs that are associated with survey design, pretesting, survey administration, follow-up with nonresponders, and data analysis. The per-reply cost of surveys administered by electronic methods has been estimated to be half the per-reply cost of postal surveys, with these values increasing with an increased response rate²⁵. This can be attributed to the extra costs of stationery, postage, administration, and follow-up with postal administration²³. If Internet programming and a domain name are used for an Internet survey, this will lead to increased costs as well; however, the cost is reduced with an increased sample size¹³.

Ethical Considerations

Before commencing a survey of orthopaedic surgeons, the research should be approved and monitored by an ethics review committee⁸. Many academic institutions have nonmedical ethics review boards where surveys not involving patients are reviewed. Informed consent is an important issue in research. Ethical guidelines of survey research emphasize that orthopaedic surgeons should volunteer participation and that the return of the completed questionnaire reflects their implied consent⁸. Since there is no need to send a separate consent form, the cover letter and survey materials should include sufficient information for the respondent to make an informed decision¹⁷. Participants should be assured of anonymity and confidentiality and be given information so that they can contact the researcher, know the purpose of the study, and know who the sponsors of the research are^8,14. Methods of maintaining confidentiality include the use of codes as well as the omission of identifying information in both the database and in subsequent publications¹⁴.

Summary

The use of a survey directed at orthopaedic surgeons can be an informative contribution to the literature. The design of the survey should be considered carefully, as a poorly designed survey may lead to inaccurate data and incorrect conclusions. A major cause of decreased validity of these surveys is the low response rate by surgeons. The different modes of survey administration all obtain responses, with postal mail receiving the highest number of responses and electronic methods receiving responses more quickly. Methods to increase response rates are necessary and may include the use of a cover letter, pretesting, appropriate survey packaging, and the mixed-mode method, in which various modes of contact are used to contact a nonresponder multiple times. Image Not Available

References

Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P. Does flattery work? A comparison of 2 different cover letters for an international survey of orthopedic surgeons. Can J Surg.2006;49:90-5.4990 2006 [PubMed]

Moses SH, Clark TJ. Effect of prize draw incentive on the response rate to a postal survey of obstetricians and gynaecologists: a randomised controlled trial [ISRCTN32823119]. BMC Health Serv Res.2004;4:14.414 2004 [CrossRef]

Zelnio RN. Data collection techniques: mail questionnaires. Am J Hosp Pharm.1980;37:1113-9.371113 1980

Bhandari M, Devereaux PJ, Swiontkowski MF, Schemitsch EH, Shankardass K, Sprague S, Guyatt GH. A randomized trial of opinion leader endorsement in a survey of orthopaedic surgeons: effect on primary response rates. Int J Epidemiol.2003;32:634-6.32634 2003 [CrossRef]

Hocking JS, Lim MS, Read T, Hellard M. Postal surveys of physicians gave superior response rates over telephone interviews in a randomized trial. J Clin Epidemiol.2006;59:521-4.59521 2006 [CrossRef]

Jiwa M, Coker AE, Bagley J, Freeman J, Coleman M. Surveying general practitioners: a new avenue. Curr Med Res Opin.2004;20:319-24.20319 2004 [CrossRef]

Maheux B, Legault C, Lambert J. Increasing response rates in physicians' mail surveys: an experimental study. Am J Public Health.1989;79:638-9.79638 1989 [CrossRef]

Mailey SK. Increasing your response rate for mail survey data collection. SCI Nurs.2002;19:78-9.1978 2002

Shiono PH, Klebanoff MA. The effect of two mailing strategies on the response to a survey of physicians. Am J Epidemiol.1991;134:539-42.134539 1991

Baron G, De Wals P, Milord F. Cost-effectiveness of a lottery for increasing physicians' responses to a mail survey. Eval Health Prof.2001;24:47-52.2447 2001 [CrossRef]

Bergk V, Gasse C, Schnell R, Haefeli WE. Mail surveys: obsolescent model or valuable instrument in general practice research? Swiss Med Wkly.2005;135:189-91.135189 2005

Brehaut JC, Graham ID, Visentin L, Stiell IG. Print format and sender recognition were related to survey completion rate. J Clin Epidemiol.2006;59:635-41.59635 2006 [CrossRef]

Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, Devereaux PJ, Guyatt GH. Internet versus mailed questionnaires: a controlled comparison (2). J Med Internet Res.2004;6:e39. Corrected and republished from: J Med Internet Res. 2004;6:e39.6e39 2004 [CrossRef]

Dillman DA. Mail and Internet surveys: the tailored design method. 2nd ed. New York: John Wiley and Sons; 2000. 2000

Roberts LM, Wilson S, Roalfe A, Bridge P. A randomised controlled trial to determine the effect on response of including a lottery incentive in health surveys [ISRCTN32203485]. BMC Health Serv Res.2004;4:30.430 2004 [CrossRef]

Asch DA, Christakis NA. Different response rates in a trial of two envelope styles in mail survey research. Epidemiology.1994;5:364-5.5364 1994 [CrossRef]

Jones D, Story D, Clavisi O, Jones R, Peyton P. An introductory guide to survey research in anaesthesia. Anaesth Intensive Care.2006;34:245-53.34245 2006

Jepson C, Asch DA, Hershey JC, Ubel PA. In a mailed physician survey, questionnaire length had a threshold effect on response rate. J Clin Epidemiol.2005;58:103-5.58103 2005 [CrossRef]

Mullner RM, Levy PS, Byre CS, Matthews D. Effects of characteristics of the survey instrument on response rates to a mail survey of community hospitals. Public Health Rep.1982;97:465-9.97465 1982

McMahon SR, Iwamoto M, Massoudi MS, Yusuf HR, Stevenson JM, David F, Chu SY, Pickering LK. Comparison of e-mail, fax, and postal surveys of pediatricians. Pediatrics.2003;111(4 Pt 1):e299-303.111e299 2003 [CrossRef]

Mavis BE, Brocato JJ. Postal surveys versus electronic mail surveys. The tortoise and the hare revisited. Eval Health Prof.1998;21:395-408.21395 1998 [CrossRef]

Sequin R, Godwin M, MacDonald S, McCall M. E-mail or snail mail? Randomized controlled trial on which works better for surveys. Can Fam Physician.2004;50:414-9.50414 2004

Raziano DB, Jayadevappa R, Valenzula D, Weiner M, Lavizzo-Mourey R. E-mail versus conventional postal mail survey of geriatric chiefs. Gerontologist.2001;41:799-804.41799 2001

Braithwaite D, Emery J, De Lusignan S, Sutton S. Using the Internet to conduct surveys of health professionals: a valid alternative? Fam Pract.2003;20:545-51.20545 2003 [CrossRef]

Jones R, Pitt N. Health surveys in the workplace: comparison of postal, email and World Wide Web methods. Occup Med (Lond).1999;49:556-8.49556 1999 [CrossRef]

VanDenKerkhof EG, Parlow JL, Goldstein DH, Milne B. In Canada, anesthesiologists are less likely to respond to an electronic, compared to a paper questionnaire. Can J Anaesth.2004;51:449-54.51449 2004 [CrossRef]

Dillman DA. Why choice of survey mode makes a difference. Public Health Rep.2006;121:11-3.12111 2006

McKenzie-McHarg K, Tully L, Gates S, Ayers S, Brocklehurst P. Effect on survey response rate of hand written versus printed signature on a covering letter: randomised controlled trial [ISRCTN67566265]. BMC Health Serv Res.2005;5:52.552 2005 [CrossRef]

Streiff MB, Dundes L, Spivak JL. A mail survey of United States hematologists and oncologists: a comparison of business reply versus stamped return envelopes. J Clin Epidemiol.2001;54:430-2.54430 2001 [CrossRef]

Shahar E, Bisgard KM, Folsom AR. Response to mail surveys: effect of a request to explain refusal to participate. The ARIC Study Investigators. Epidemiology.1993;4:480-2.4480 1993 [CrossRef]

Ward J, Bruce T, Holt P, D'Este K, Sladden M. Labour-saving strategies to maintain survey response rates: a randomised trial. Aust N Z J Public Health.1998;22(3 Suppl):394-6.22394 1998 [CrossRef]

Clark TJ, Khan KS, Gupta JK. Provision of pen along with questionnaire does not increase the response rate to a postal survey: a randomised controlled trial. J Epidemiol Community Health.2001;55:595-6.55595 2001 [CrossRef]

Asch DA, Christakis NA, Ubel PA. Conducting physician mail surveys on a limited budget. A randomized trial comparing $2 bill versus $5 bill incentives. Med Care.1998;36:95-9.3695 1998 [CrossRef]

Carling C. International questionnaire postal response rate: an experiment comparing no return postage to provision of International Postage Vouchers—"Coupon-Réponse International". BMC Health Serv Res.2004;4:16.416 2004 [CrossRef]

Topics

attitude ; esthetics ; health personnel ; internet ; telephone ; anonymity ; incentives ; health care ; cover letter

Username

Password

Access for Patients

Forgot Password?

Have a subscription to the print edition?

Not a Subscriber?

Buy this Article

Get online access for 30 days for $35

New to JBJS?

Subscribe to the online edition for 30 days for $75

Register for a FREE limited account to get full access to all CME activities, to comment on public articles, or to sign up for alerts.

Have a subscription to the print edition?

Current subscribers to The Journal of Bone & Joint Surgery in either the print or quarterly DVD formats receive free online access to JBJS.org.

Activate your online account now

Forgot your password?

Enter your username and email address. We'll send you a reminder to the email address on record.

Username:*

Email Address:*

Forgot your username or need assistance? Please contact customer service at subs@jbjs.org. If your access is provided
by your institution, please contact you librarian or administrator for username and password information. Institutional
administrators, to reset your institution's master username or password, please contact subs@jbjs.org

References

Accreditation Statement

These activities have been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Academy of Orthopaedic Surgeons and The Journal of Bone and Joint Surgery, Inc. The American Academy of Orthopaedic Surgeons is accredited by the ACCME to provide continuing medical education for physicians.

CME Activities Associated with This Article

Submit a Comment

Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discretion of JBJS editorial staff.

* = Required Field

Comment Author(s) * (if multiple authors, separate names by comma)

Example: John Doe

Affiliation & Institution*

Comment Title*

Comment*

Cancel

Print
PDF
Email

Recipient(s) will receive an email with a link (good for 5 days) to 'Survey Design in Orthopaedic Surgery: Getting Surgeons to Respond'.
Your Name:*
Example: John Doe

Email Address:* CC Me:
Enter your valid email address. Example: jdoe@example.com

Recipient's Email Address:*

Separate multiple email address with semi-colons (up to 5).

Subject:* 's The Journal of Bone & Joint Surgery: 'Survey Design in Orthopaedic Surgery: Getting Surgeons to Respond'
Subject for your email.

Message:

(Optional, message will truncate at 1000 characters)

Processing your request... Please Wait...

Copyright © in the material you requested is held by The Journal of Bone & Joint Surgery, Inc. (unless otherwise noted). This email ability is provided as a courtesy, and by using it you agree that that you are requesting the material solely for personal, non-commercial use, and that it is subject to Journal of Bone & Joint Surgery, Inc.'s Terms of Use. The information provided in order to email this topic will not be used to send unsolicited email, nor will it be furnished to third parties. Please refer to The Journal of Bone & Joint Surgery Inc.'s Privacy Policy for further information.

Copyright © The Journal of Bone & Joint Surgery Inc. All rights reserved.
Share
Get Citation

Sheila Sprague, Laura Quigley, Mohit Bhandari; Survey Design in Orthopaedic Surgery: Getting Surgeons to Respond. The Journal of Bone & Joint Surgery. 2009 May;91(Supplement_3):27-34.

Download citation file:

RIS (Zotero)

EndNote

BibTex

Medlars

ProCite

RefWorks

Reference Manager

Copyright © The Journal of Bone and Joint Surgery, Inc.
Rights & Permissions
Article Alerts

Processing your request... Please Wait.
Submit a Comment