Deck 5: Identifying Good Measurement

Question 1

Which of the following is NOT an example of physiological measurement?

A) Measurements of hormones in the bloodstream
B) Blood pressure measurements
C) Number of panic attacks a patient reports
D) A brain scan made using an fMRI
E) All of the above are examples of physiological measurement

Answer 1

Number of panic attacks a patient reports

Question 2

RESEARCH STUDY 5.1
Dr. Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Mayo scale. Question 1 reads, "I tend not to think about other people as much as myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."
Refer to Research Study 5.1 above to answer the following question. Dr.Valencia is concerned whether her measure will really measure narcissism or if it will measure some other related concept.She is concerned about the scale's ____________.

A) Conceptual definition
B) Operational definition
C) Validity
D) Reliability
E) Convenience

Answer 2

Validity

Question 3

RESEARCH STUDY 5.1
Dr. Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Mayo scale. Question 1 reads, "I tend not to think about other people as much as myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."
Refer to Research Study 5.1 above to answer the following question. Before using the measure in her study,Dr.Valencia analyzes the data she gets from her students.She looks for the relationship between each of the individual questions.She sees that participants who agree with Question 1 also agree with Question 3 and disagree with Question 2.This is a test of which of the following?

A) Interrater reliability
B) Internal reliability
C) Test-retest reliability
D) Construct reliability
E) None of the above

Answer 3

Internal reliability

Question 4

Which of the following is possible?

A) A measure is neither reliable nor valid.
B) A measure is both valid and reliable.
C) A measure is reliable but not very valid.
D) Only a and b are possible.
E) All of the above are possible.

Question 5

What is the difference between a ratio scale of measurement and an interval scale of measurement?

A) A ratio scale of measurement has a zero value that actually means "nothing" or "the absence of something," but an interval scale does not.
B) An interval scale of measurement is a type of measurement used for categorical measurements, but a ratio scale is used for quantitative measurements.
C) An interval scale has equal intervals, but a ratio scale does not.
D) A ratio scale of measurement cannot be used to compare people's scores, but interval scales can (e.g., Phillip is twice as fast).
E) There is no difference between the two scales of measurement.

Question 6

Felipe plans to watch how positively or negatively teachers treat children in their classrooms who have ADHD for his research methods class.This is an example of what type of measurement?

A) Self-report measurement
B) Observational measurement
C) Physiological measurement
D) Archival measurement
E) All of the above are variables

Question 7

Hosea is studying the relationship between caffeine consumption and problem-solving ability.Which of the following is a quantitative way to operationalize problem-solving ability?

A) The time spent solving a math problem
B) The type of puzzle solved (Sudoku puzzle or a crossword puzzle)
C) Whether participants used insight or trial-and-error techniques to solve the problem
D) The report of a teacher about whether a student is a good or bad problem solver
E) All of the above are quantitative ways of operationalizing caffeine consumption

Question 8

RESEARCH STUDY 5.1
Dr. Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Mayo scale. Question 1 reads, "I tend not to think about other people as much as myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."
Refer to Research Study 5.1 above to answer the following question. Before using the measure in her study,Dr.Valencia gives the measure to a group of students on Tuesday.She gives the measure to them again on Thursday.Dr.Valencia is examining the scatterplot of the data she collected on Day 1 and Day 2.On the scatterplot,she sees that the dots are very close to the line.This indicates which of the following?

A) A strong relationship
B) A negative relationship
C) A nonrelationship
D) A valid finding
E) A reliable finding

Question 9

Asking many similar questions when trying to measure a concept is done to:

A) Ensure validity
B) Cancel out random errors
C) Make sure participants pay attention
D) Allow participants to skip questions
E) Make a better scatterplot

Question 10

How many subcategories of quantitative variables exist?

A) Two
B) Three
C) Four
D) Five
E) Six

Question 11

RESEARCH STUDY 5.1
Dr. Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Mayo scale. Question 1 reads, "I tend not to think about other people as much as myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."
Refer to Research Study 5.1 above to answer the following question. Dr.Valencia is concerned about the validity of the measure of narcissism recommended by her colleague.She sends a copy of the measure to the faculty members in her psychology department to look at and they all tell her it looks like it will measure narcissism.She now has evidence of which of the following?

A) Predictive validity
B) Content validity
C) Face validity
D) Discriminant validity
E) Concurrent validity

Question 12

Todd is also studying the effect of popularity on academic success for his research methods project.He decides to measure popularity by asking each elementary school student to tell him how many friends he or she has.He assumes that more friends means the student is more popular.Which of the following best describes this variable?

A) A ratio scale of measurement
B) A quantitative variable
C) A self-report measure
D) Both b and c
E) All of the above

Question 13

RESEARCH STUDY 5.1
Dr. Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Mayo scale. Question 1 reads, "I tend not to think about other people as much as myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."
Refer to Research Study 5.1 above to answer the following question. Dr.Valencia calculates a correlation coefficient (r)to examine the relationship between Question 1 and Question 2 and between Question 1 and Question 3.She finds a correlation coefficient of r ? -0.73 between Questions 1 and 2 and a correlation coefficient of r ? 0.74 between Questions 1 and 3.Which of the following is true of her findings?

A) There appears to be good internal reliability in the scale.
B) The correlation between Questions 1 and 2 and Questions 1 and 3 are in the same direction.
C) The correlation between Questions 1 and 2 is much weaker than between Questions 1 and 3.
D) The negative correlation between Question 1 and Question 2 is bad for Dr. Valencia.
E) All of the above are true.

Question 14

Hosea is studying the relationship between caffeine consumption and problem-solving ability.Which of the following is a categorical way to operationalize caffeine consumption?

A) The number of glasses of soda consumed in a day
B) The number of milligrams of caffeine consumed during the study
C) The frequency of buying coffee drinks
D) Whether the participant drank a soda in the 24 hours prior to the study
E) All of the above are categorical ways of operationalizing caffeine consumption

Question 15

RESEARCH STUDY 5.1
Dr. Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Mayo scale. Question 1 reads, "I tend not to think about other people as much as myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."
Refer to Research Study 5.1 above to answer the following question. Before using the measure in her study,Dr.Valencia gives the measure to a group of students on Tuesday.She gives the measure to them again on Thursday.She then compares the scores between the two days.This is a test of which of the following?

A) Interrater reliability
B) Internal reliability
C) Test-retest reliability
D) Construct reliability
E) None of the above

Question 16

Serena plans to interview several teachers about their attitude toward teaching children who have attention deficit hyperactivity disorder (ADHD)for her research methods class.This is an example of what type of measurement?

A) Self-report measurement
B) Observational measurement
C) Physiological measurement
D) Archival measurement
E) All of the above are variables

Question 17

A correlation-based statistic called __________ is commonly used to determine internal reliability.

A) Cronbach's alpha
B) Kappa
C) A scatterplot
D) Pearson's r
E) None of the above

Question 18

Which of the following is true of scatterplots?

A) They can be used to examine interrater reliability.
B) They can be used to examine internal reliability.
C) They are a less common way to examine test-retest reliability compared with correlation coefficients.
D) Both a and b are true.
E) All of the above are true.

Question 19

Naomi is studying the effect of popularity on academic success for her research methods project.To do this,she has elementary school students rate how popular each member of their class is.She then uses this information to rank the students on popularity (e.g.,John is the most popular,Vanessa is the second-most popular).Which of the following best describes this variable?

A) An ordinal scale of measurement
B) A self-report measurement
C) A categorical measurement
D) Both a and b
E) Both b and c

Question 20

Which of the following is true of operational definitions?

A) There are multiple operational definitions that are possible for any one conceptual definition.
B) The specification of operational definitions is one of the creative aspects of the research process.
C) Operational definitions are created after conceptual definitions are determined.
D) Both a and b are true.
E) All of the above are true.

Question 21

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. To test his measure,Dr.Sheffield gives his measure to a group of people in GA and another group in AA.He finds that people in the GA group have higher scores on his new measure than people in the AA group.This procedure is known as a:

A) Test-retest paradigm
B) Known-groups paradigm
C) Prediction paradigm
D) Group evaluation paradigm
E) None of the above

Question 22

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. To test his measure,Dr.Sheffield gives his measure to a group of his clients and at the same time measures how many times they have been gambling in the past month.He predicts that clients who score higher on his measure will also report gambling more times in the past month.This procedure is meant to provide evidence for which of the following?

A) Face validity
B) Content validity
C) Predictive validity
D) Concurrent validity
E) Discriminant validity

Question 23

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. Dr.Sheffield gives his measure to his supervisor,who is also an expert in pathological gambling.His supervisor says that his measure appears to test all the components of pathological gambling,including feeling restless when attempting to stop gambling,jeopardizing jobs in order to keep gambling,and using gambling to escape from problems and a bad mood.Given this information,Dr.Sheffield's measure has evidence of which of the following?

A) Face validity
B) Content validity
C) Predictive validity
D) Concurrent validity
E) Discriminant validity

Question 24

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. Dr.Sheffield decides to test the predictive validity of his measure.Dr.Sheffield gives his measure to a group of people that includes suspected problem gamblers and non-gamblers.Which of the following options below does he need to do in addition to get evidence for predictive validity?

A) Give the measure to a group of people attending Gamblers Anonymous meetings
B) Two months later, ask the same group of people to report how many times they have gambled recently
C) Ask the participants to give their opinion on whether the measure is valid
D) Give a measure of alcohol addiction to the same group of clients
E) None of the above

Question 25

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. If Dr.Sheffield's measure does not actually measure pathological gambling,his measure is said to lack which of the following?

A) Validity
B) Reliability
C) Conceptualization
D) Operationalization
E) None of the above

Question 26

Which of the following is true of the difference between predictive validity and concurrent validity?

A) Predictive validity is supported when a measure predicts a future outcome rather than a current outcome.
B) Concurrent validity is tested using a correlation coefficient rather than a scatterplot.
C) Predictive validity is tested with Cronbach's alpha rather than kappa.
D) Concurrent validity is tested after predictive validity is established.
E) All of the above are differences between predictive validity and concurrent validity.

Question 27

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. Dr.Sheffield has now decided that he wants to test his measure on some university students (who some estimates say have a 6% prevalence rate of compulsive gambling).He has a group of 100 university students complete his measure.He also has them complete two other measures (one that measures addictive behavior in general and one that measures general attitudes toward gambling).He finds that his new measure is positively associated with each of these other measures.This procedure has provided evidence for the _______________ of Dr.Sheffield's measure.

A) Content validity
B) Predictive validity
C) Convergent validity
D) Concurrent validity
E) Discriminant validity

Question 28

Explain why your textbook argues,"In fact,operationalizations are one place where creativity comes into the research process."

Question 29

Why are convergent and discriminant validity often evaluated together?

A) Because both terms mean the same thing
B) Because both involve collecting information from a lot of psychological measures of theoretical interest
C) Because both must be established before examining predictive and concurrent validity
D) Because both require the use of scatterplots
E) Because both have to be determined by a panel of experts

Question 30

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. Dr.Sheffield has decided to test the discriminant validity of his new measure.He has a group of first-time GA attendants complete his measure,and finds that they score higher than a group of people who do not attend the group.Which of the following results would provide evidence for discriminant validity?

A) He finds that the GA attendees score higher on his measure than the non-GA attendees.
B) He finds that the GA attendees score higher on his measure than the non-GA attendees, and they also score higher than a group of people attending Alcoholics Anonymous meetings (who are addicted to something different).
C) He finds that more recent GA joiners score higher than veteran GA attendees, who are more recovered.
D) He finds that the measure he used is also associated with people's past diagnoses of pathological gambling.
E) None of the above provide evidence.

Question 31

RESEARCH STUDY 5.2
Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions and it takes 20 minutes to complete.
Refer to Research Study 5.2 above to answer the following question. To test his measure,Dr.Sheffield gives his measure to a group of people in Gamblers Anonymous (GA)and another group in Alcoholics Anonymous (AA).He finds that people in the GA group have higher scores on his new measure than people in the AA group.Why did Dr.Sheffield do this?

A) To obtain evidence for face validity
B) To obtain evidence for content validity
C) To obtain evidence for convergent validity
D) To obtain evidence for concurrent validity
E) To obtain evidence for internal validity

Question 32

In which of the following ways are content and face validity similar?

A) Both involve subjective judgments.
B) Both involve judgments based on experts.
C) Both are preferred by psychologists.
D) Both a and b are true.
E) All of the above are true.

Question 33

Explain why many people feel that physiological measurements are the best way to operationalize a variable but why they may not,in fact,be the best.

Question 34

Imagine that you are trying to measure people's stress.Create a quantitative definition of stress that is ordinal,interval,and ratio.

Question 35

RESEARCH STUDY 5.3:
Lauren, Sarah, and Jennifer are students in Dr. Shaffer's Research Methods class. For a class assignment, they are asked to devise an operational definition for romantic attachment, or love. Lauren decides to ask a group of married couples to report if they are in love with their spouse (1 indicates "Yes, I am in love" and 2 indicates "No, I am not in love"). Sarah decides to watch couples sitting on campus and measure how close they sit together, assuming that people who sit closer together are more in love. Jennifer decides to recruit a group of newlyweds and measure the change in heart rate that occurs between the start of the study (when the person is alone) and the moment they see their spouse walk into the room.
Refer to Research Study 5.3 above to answer the following question.
For each student described above,indicate whether the operational definition of her variable is categorical or quantitative.If a student's operational variable is categorical,explain whether it could or could not be made quantitative.

Question 36

Another word for discriminant validity is ___________ validity.

A) Convergent
B) Asymmetrical
C) Predictive
D) Multiple
E) None of the above

Question 37

An educational psychologist is testing the discriminant validity of a new measure of numerical learning difficulties.He gives his measure to a group of students along with another measure of verbal learning difficulties,which he predicts should not be strongly related to numerical learning difficulties.Which of the following correlations would the psychologist hope to find in order to establish discriminant validity?

A) r = 1.0
B) r = -1.0
C) r = 0.83
D) r = -0.18
E) Any of the above would be acceptable

Question 38

Establishing construct validity is most important for which of the following?

A) A concrete construct
B) A behavior that is directly observable
C) Physical measurements (e.g., length)
D) An abstract concept
E) All of the above

Question 39

Imagine that you are trying to measure people's stress.Provide an example of measuring stress using a self-report measure,a physiological measure,and a behavioral measure.

Question 40

Establishing construct validity would probably be most important for which of the following?

A) A measure of blood pressure
B) A measure of the number of seizures a person has per week
C) A measure of religiosity
D) A measure of obesity
E) All of the above

Question 41

Dr.Sarolli is planning on measuring literacy (defined as the ability to read and write written text in one's native language).He decides he wants to create a quantitative operational definition.Create an ordinal,interval,and ratio way to measure literacy.

Question 42

RESEARCH STUDY 5.3:
Lauren, Sarah, and Jennifer are students in Dr. Shaffer's Research Methods class. For a class assignment, they are asked to devise an operational definition for romantic attachment, or love. Lauren decides to ask a group of married couples to report if they are in love with their spouse (1 indicates "Yes, I am in love" and 2 indicates "No, I am not in love"). Sarah decides to watch couples sitting on campus and measure how close they sit together, assuming that people who sit closer together are more in love. Jennifer decides to recruit a group of newlyweds and measure the change in heart rate that occurs between the start of the study (when the person is alone) and the moment they see their spouse walk into the room.
Refer to Research Study 5.3 above to answer the following question.
For each student described above,indicate whether the operational definition of her variable is self-reported,observational,or physiological.

Question 43

RESEARCH STUDY 5.4
Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."
Refer to Research Study 5.4 above to answer the following question.
Provide a suggestion for how Dr.Li could establish the predictive validity of his measure and a suggestion for how he could establish the concurrent validity of his measure.

Question 44

RESEARCH STUDY 5.4
Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."
Refer to Research Study 5.4 above to answer the following question.
Dr.Li gives his new measure to a group of 100 adults in order to test the internal validity of his measure.Draw two scatterplots: one that demonstrates the likely relationship between Statement 1 and Statement 2 and one that demonstrates the likely relationship between Statement 1 and Statement 3.Explain why you have drawn each scatterplot as you have.

Question 45

Name the three types of reliability and the four types of validity.

Question 46

RESEARCH STUDY 5.4
Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."
Refer to Research Study 5.4 above to answer the following question.
Given the above situation,explain the difference between predictive validity,convergent validity,and discriminant validity.Why is each one important?

Question 47

Explain the difference between validity and reliability.Explain whether they are related or independent concepts.

Question 48

RESEARCH STUDY 5.4
Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."
Refer to Research Study 5.4 above to answer the following question.
Name two types of reliability and how each one could be established,given the above scenario.

Question 49

RESEARCH STUDY 5.4
Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."
Refer to Research Study 5.4 above to answer the following question.
Dr.Li gives his new measure to a group of 100 adults in order to test the internal validity of his measure.What type of correlation coefficients (r)would Dr.Li expect to find between Statement 1 and Statement 2 and between Statement 1 and Statement 3? What would these correlation coefficients tell Dr.Li about the internal reliability of his measure?

Question 50

RESEARCH STUDY 5.4
Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."
Refer to Research Study 5.4 above to answer the following question.
Provide a suggestion for how Dr.Li could establish the face validity of his measure and a suggestion for how he could establish the content validity of his measure.