Deck 9: How Do We Construct and Administer Surveys and Use Survey Data

Question 1

Companies that specialize in constructing, administering, and analyzing survey data are called ______.

A) survey research firms
B) survey consulting firms
C) opinion pollsters
D) survey companies

Answer 1

A

Question 2

Joe designs and conducts surveys and analyzes survey results. According to your text, Joe is a ______.

A) survey researcher
B) survey specialist
C) project analyst
D) scientific method researcher

Answer 2

A

Question 3

The results of psychological tests are often reported ______, and the results of surveys are often reported ______.

A) as an overall derived score; as an overall derived score
B) at the question level; at the question level
C) as an overall derived score; at the question level
D) at the question level; as an overall derived score

Answer 3

C

Question 4

If you were going to conduct a study that involved examining a cause-and-effect relationship between two or more variables, what type of research technique would you most likely use?

A) experimental
B) scientific
C) descriptive
D) empirical

Question 5

Which one of the following is FALSE about surveys?

A) Survey development is as much a science as it is an art.
B) All good surveys are developed following the scientific method.
C) Reliability and validity are important issues when developing surveys.
D) Constructing surveys is a simple and quick process if the scientific method is followed.

Question 6

According to your textbook, which one of the following is a well-known survey software provider?

A) Snap Surveys
B) Inquisite
C) StatPac
D) Qualtrics

Question 7

A table that lists objectives, operational definitions for each objective, and the number of questions that measure each objective of a survey is very similar to a ______.

A) research method
B) test specification table
C) reliability table
D) validation matrix

Question 8

______ research techniques help us to determine cause and effect, and ______ research techniques help us to describe a situation or a phenomenon.

A) Empirical; scientific
B) Scientific; empirical
C) Descriptive; experimental
D) Experimental; descriptive

Question 9

What statement about surveys is FALSE?

A) Simple survey questions are easy to develop.
B) Surveys are a science because science is a process.
C) Survey research is an art.
D) Constructing a reliable and valid survey takes time.

Question 10

How many steps are associated with a scientific approach to survey design?

A) 3
B) 5
C) 7
D) 9

Question 11

Survey objectives refer to the ______.

A) need for the survey and what the survey will measure
B) need for the survey and the purpose of the survey
C) purpose of the survey and operationally defined objectives
D) purpose of the survey and what the survey will measure

Question 12

Whereas psychological tests focus on ______ outcomes, surveys focus on ______ outcomes.

A) group; individual
B) individual; group
C) specific; broad
D) broad; specific

Question 13

A class evaluation survey administered at the end of a semester is what type of research tool?

A) intuitive
B) rational
C) experimental
D) descriptive

Question 14

Which one of the following is NOT a characteristic of a good survey?

A) evidence of reliability and validity
B) specific and measurable objectives
C) contains a minimum of 2 survey items per objective
D) includes accurate reporting of written and verbal results

Question 15

According to your textbook, what is the most popular type of survey software?

A) software purchased at a store selling electronics
B) software available on a company's server and accessed over the Internet
C) software accessed over the Internet and downloaded onto a personal computer
D) software purchased through educational institutions at discounted rates

Question 16

What is the fourth step in a scientific approach to survey design?

A) construct the survey
B) analyze the survey data
C) administer the survey
D) communicate the survey findings

Question 17

If you were going to design a survey, what would be the first step?

A) specify the people who will respond to the survey
B) state a hypothesis
C) identify the objectives of the survey
D) define the objectives operationally

Question 18

If you were going to conduct a study that involved explaining a phenomenon, what type of research technique would you most likely use?

A) experimental
B) scientific
C) descriptive
D) empirical

Question 19

Each of the objectives of a survey must be ______.

A) rationalized
B) reliable and valid
C) operationalized
D) objectified

Question 20

Whereas the results of psychological tests are often reported at the ______ level, the results of surveys are often reported at the ______ level.

A) individual; group
B) group; individual
C) basic; advanced
D) advanced; basic

Question 21

If you were observing workers performing a task and recording the results, you be completing a ______.

A) structured observation
B) structured record review
C) personal interview
D) individually administered survey

Question 22

Why is the following survey question inappropriate? "Did you enjoy watching and listening to the CD of the concert?"

A) The question is not specific about which concert.
B) The question is not in context.
C) The question is a double-barreled question.
D) The question is not in correct syntax.

Question 23

Imagine you are preparing a survey instrument. Which one of the following should you NOT include in your survey?

A) headings and subheadings to guide the respondent
B) bold typeface to emphasize key points
C) casual font to capture the attention of the respondents
D) adequate white space to make the survey look more inviting

Question 24

If you were stopped in a shopping mall and asked to complete a survey in the presence of the individual who stopped you, what type of survey would you be completing?

A) mail survey
B) self-administered survey
C) personal interview
D) individually administered survey

Question 25

When writing survey questions, it is important ensure that there is a clear relationship between each survey question and the ______.

A) final outcome
B) data analysis
C) respondent's answers
D) survey's objectives

Question 26

If you were reviewing student research papers looking for specific information and recording the results, you be completing a ______.

A) structured observation
B) structured record review
C) personal interview
D) individually administered survey

Question 27

Which one of the following guidelines did your textbook authors NOT suggest when printing a survey?

A) single spacing, eliminating the cover page
B) stapling in the upper left corner for longer surveys
C) spiral binding or printing in booklet format for extremely long surveys
D) printing on two sides of a sheet of paper for surveys less than three pages

Question 28

Which one of the following would you typically NOT include in a survey plan?

A) estimated costs for developing and administering the survey, and analyzing the survey data
B) the phases and steps necessary for completing the survey development
C) a timeline for completing each phase of the survey
D) operational definitions of the objectives the number of questions needed to meet the survey objectives

Question 29

What rating scale would be most appropriate for responding to a survey question that asks, "How well did the product meet your expectations?"

A) A: poor; B: below average; C: average; D: above average; E: excellent
B) A: disagree; B: disagree somewhat; C: agree somewhat; D: agree altogether
C) A: not at all; B: somewhat; C: mostly; D: all
D) A: did not meet expectations; B: met expectations somewhat; C: mostly met expectations; D:met all expectations

Question 30

Why is "Do you drink a lot?" an unacceptable survey question?

A) It is not concrete and clear.
B) It may be biased for race or sex.
C) It is not in correct syntax.
D) It is too private and sensitive.

Question 31

When developing a survey, what should the survey researcher begin each group of survey questions with?

A) a simple question
B) a definition of the topic of the questions
C) an admonition to answer correctly and honestly
D) an explanation of the questions' purpose

Question 32

You are preparing the front page of a survey you developed. Which one of the following should you include on the front page?

A) title, logo, appeal, and instructions
B) title and instructions
C) title and logo
D) appeal and instructions

Question 33

When should a survey researcher include an open-ended survey question?

A) anytime they want to
B) when they do not plan to use numerical responses
C) when they cannot predict possible responses
D) when the survey objectives are general and not specific

Question 34

What are nonsampling measurement errors?

A) errors resulting from respondents not being chosen randomly from the population
B) errors resulting from respondents failing to answer questions or return the survey
C) errors that occur during analysis and interpretation of survey data
D) errors that occur because of poor survey design or administration

Question 35

Which one of the following is NOT one of the 6 recommended items to include in survey instructions?

A) who is sponsoring or conducting the survey
B) importance of completing the survey to the best of ability
C) how respondents will be compensated for their time
D) a thank you to respondents for their cooperation

Question 36

If you were stopped in a shopping mall and asked to complete a survey involving direct contact between the survey researcher and you, what type of survey would you be completing?

A) mail survey
B) self-administered survey
C) personal interview
D) individually administered survey

Question 37

What rating scale would be most appropriate for responding to a survey question that asks, "How much time should a student study each week for an undergraduate college course?"

A) A: not much; B: a little; C: an average amount; D: a lot; E: almost always
B) A: 0-30 min; B: 31-60 min; C: 61-120 min; D: more than 120 min
C) A: 0-1 hr; B: 1-2 hr; C: 2-3 hr; D: more than 3 hr
D) A: below average; B: average; C: above average; D: no opinion

Question 38

Survey researchers should write questions at what readability level?

A) moderate readability level
B) readability level of the survey population
C) low readability level
D) high readability level

Question 39

What type of question is the following: "If the organization where you work offered netiquette workshops, would you attend?"

A) free choice
B) semantic differential
C) forced choice
D) ranking

Question 40

What type of question is the following: "Please circle the number that best represents how you are currently feeling."
$$\begin{array} { | c | c | c | c | c | } \hline \text { Happy } & & & & \text { Sad } \\\hline 1 & 2 & 3 & 4 & 5 \\\hline\end{array}$$

A) free choice
B) semantic differential
C) forced choice
D) ranking

Question 41

What sampling technique is a researcher using when a population is divided into subgroups based on one or more characteristics and a random sample is selected from each group?

A) simple random sampling
B) systematic random sampling
C) cluster sampling
D) stratified random sampling

Question 42

What is a sample?

A) a group of respondents who may receive the survey
B) a representative subset of the population
C) those respondents who return the survey
D) all respondents in the target audience

Question 43

When pretesting a survey, you ask respondents to describe their thoughts when answer survey questions. What method are you using?

A) concurrent "think-aloud" interview
B) retrospective "think-aloud" interview
C) paraphrasing
D) confidence ratings

Question 44

What does homogeneity of the population refer to?

A) how similar the people in a population are to one another
B) how similar the respondents to the survey are to one another
C) the extent to which the target audience is similar to the population
D) the extent to which the sample represents the population

Question 45

Which one of the following statement about simple random samples is TRUE?

A) The sample will have the same characteristics as the population.
B) The sample will be representative of the population.
C) The sampling technique is a form of probability sampling.
D) The sample is drawn by identifying every nth member of the population.

Question 46

Which one of the following is TRUE about simple random sampling?

A) Every respondent has a 95% probability of being chosen in the sample.
B) Every respondent has an equal probability of completing the survey.
C) Every member of a sample has an equal chance of completing the survey.
D) Every member of a population has an equal chance of being chosen for the sample.

Question 47

What is a population?

A) those in the target audience who receive a survey
B) those in the target audience who return a survey
C) all members of a survey's target audience
D) all members of the group who return a survey

Question 48

What is the purpose of behavior coding?

A) to classify respondents based on their behaviors
B) to identify the types and frequencies of interviewer and respondent behaviors
C) to inform respondents of their rights and responsibilities
D) to improve communication among multiple researchers when they interview groups

Question 49

If you were interested in designing a survey to find out how college seniors feel about the usefulness of the Graduate Record Exam (GRE) as a graduate admissions test, the population would be all college seniors who ______.

A) plan to attend graduate school
B) have completed the GRE
C) completed the GRE with scores high enough to apply for graduate school
D) completed the GRE and plan to attend graduate school

Question 50

After developing a survey, it is a good idea to pretest the survey for all EXCEPT which one of the following reasons?

A) to identify sources of sampling measurement errors
B) to examine the effectiveness of revisions to a question or an entire survey
C) to examine the effect of alternative versions of a question or survey
D) to assess the final version of a survey for ease of completion

Question 51

To select a random sample, what could a researcher do?

A) take every nth person from a list of everyone in the population
B) use cluster sampling to ensure the sample is representative
C) use a table of random numbers
D) ask respondents who wish to be in the sample to volunteer

Question 52

What do we call the process of administering a survey to a large representative group of individuals to determine any problems with administration, item interpretation, and so on?

A) a field test
B) a focus group
C) a survey promotion
D) a behavior coding study

Question 53

When pretesting a survey, which one of the following methods would involve asking respondents to repeat questions in their own words?

A) concurrent "think-aloud" interview
B) retrospective "think-aloud" interview
C) paraphrasing
D) confidence ratings

Question 54

What would the survey response percentage be if you sent a survey to 125 people and 76 people returned the survey?

A) 61%
B) 76%
C) 100%
D) 125%

Question 55

When pretesting a survey, what method would you be using if you asked respondents the following: "On a scale from 1 to 5, please rate how confident you are that your response to this survey question is correct, where 1 is not confident at all and 5 is very confident."

A) think-aloud interviews
B) retrospective interviews
C) evaluation ratings
D) confidence ratings

Question 56

Nonprobability sampling methods are often used because they ______.

A) provide a representative sample
B) yield the most accurate data
C) are convenient
D) are reliable and valid

Question 57

How would you calculate a survey response rate?

A) count the number of people who received the survey
B) count the number of people who returned the completed survey
C) divide the number of people who received the survey by the number of people who returned the survey
D) divide the number of people who returned the survey by the number of people who received the survey

Question 58

The more ______ the members of the population, the ______ the sample that is necessary.

A) similar; larger
B) similar; smaller
C) knowledgeable; smaller
D) educated; smaller

Question 59

During a pilot test, a survey researcher identified some errors associated with the design and administration of a recently developed survey. The survey researcher identified ______.

A) nonsampling measurement errors
B) sampling measurement errors
C) true value errors
D) item response errors

Question 60

If you were field-testing two or more versions of a question to determine which version provides the most accurate results, you would be conducting a ______.

A) validity study
B) reliability study
C) behavior coding test
D) split sample test

Question 61

Experienced survey researchers would probably tell you that surveys share certain characteristics. What are those characteristics? Explain each and give an example.

Question 62

Imagine you were designing a survey following the five-step scientific approach to survey design. Explain the five steps and what you would do during each step.

Question 63

A college dean constructed a survey to measure college students' attitudes toward learning classical languages. She obtained a list of students from the registrar's office and divided the students into four lists: freshmen, sophomores, juniors, and seniors. She then divided each of the four lists into lists for men and women. Finally, she used a random numbers table to choose eight students from each list. What sampling method was she using?

A) simple random sampling
B) cluster sampling
C) systematic sampling
D) stratified random sampling

Question 64

What is the final phase of conducting a survey?

A) analyzing the survey data
B) presenting findings
C) calculating response rates
D) thanking the respondents for their participation

Question 65

What analysis does NOT belong to the category of univariate analyses?

A) frequency count
B) correlations
C) response rate
D) measures of central tendency

Question 66

What are the most common methods for gathering evidence of survey reliability/precision?

A) test-retest and alternate forms reliability
B) test-retest and split-half reliability
C) alternate forms and scorer reliability
D) alternate forms and test-retest

Question 67

o Survey question rating scales should always match the type of information being requested.
$\bullet$ Survey question should include the appropriate categorical alternatives.
o Only when you are asking a question for which you have no idea of the possible responses, survey questions should be open ended.
$\bullet$ Survey question should ask one and only one question.
o Survey questions should not be double-barreled questions--questions that are actually asking two or more questions in one.
$\bullet$ Survey question should be written at a comfortable reading level.
o Survey questions should be written with a low readability level, so that respondents will be more likely to understand and interpret questions appropriately.

-What is the difference between survey reliability/precision and survey validity?

Question 68

What are random errors?

A) errors associated with poor survey administration
B) errors that are unpredictable
C) errors that arise from poor sampling methods
D) systematic errors

Question 69

What statistic do data analysts calculate to determine the reliability of the survey?

A) coefficient alpha
B) correlation coefficient
C) validity coefficient
D) Cohen's kappa

Question 70

In survey research, survey validity refers to the degree to which the survey ______.

A) assesses the concept a researcher is measuring
B) results predict an expected outcome
C) has convergent evidence of validity
D) correlates with another similar measure

Question 71

Imagine you were asked to deliver a presentation to a group of students learning about survey design. The purpose of the presentation was to increase student understanding of some of the most popular types various types of survey questions. What types of survey questions would you talk about? What examples would you provide?

Question 72

When writing survey questions, survey developers make sure their survey questions have specific characteristics. These characteristics increase the likelihood of obtaining accurate information. For example, survey developers ensure survey questions are purposeful and straightforward. Identify and explain four additional characteristics of survey questions that you believe are most critical to obtaining accurate information. Provide an example of a survey question that demonstrates each characteristic.