Deck 5: Identifying Good Measurement

Question 1

Naomi is studying the effect of popularity on academic success for her research methods project.To do this,she has elementary school students rate how popular each member of their class is.She then uses this information to rank the students on popularity (e.g. ,John is the most popular,Vanessa is the second-most popular).Which of the following best describes this variable?

A)An ordinal scale of measurement
B)A self-report measurement
C)A categorical measurement
D)An interval scale of measurement

Answer 1

An ordinal scale of measurement

Question 2

Which of the following is NOT an example of physiological measurement?

A)Measurements of hormones in the bloodstream
B)Blood pressure measurements
C)Number of panic attacks a patient reports
D)A brain scan made using an fMRI

Answer 2

Number of panic attacks a patient reports

Question 3

For her research methods class,Serena plans to interview several teachers about their attitude toward teaching children who have attention-deficit/hyperactivity disorder (ADHD).This is an example of what type of measurement?

A)Self-report measurement
B)Observational measurement
C)Physiological measurement
D)Archival measurement

Answer 3

Self-report measurement

Question 4

How many subcategories of quantitative variables exist?

A)Two
B)Three
C)Four
D)Five

Question 5

Dr.Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic.One of her first tasks is to determine which of her participants are narcissistic and which are not.She decides to use the scale created by a colleague,the Mayo scale.Question 1 reads,"I tend not to think about other people as much as I think about myself." Question 2 reads,"I do not have a high opinion of myself." Question 3 reads,"I think other people think I am really special."

Dr.Valencia is concerned about the validity of the measure of narcissism recommended by her colleague.She sends a copy of the measure to the faculty members in her psychology department to look at and they all tell her it looks like it will measure narcissism.She now has evidence of which of the following?

A)Content validity
B)Face validity
C)Discriminant validity
D)Concurrent validity

Question 6

Which of the following is true of operational definitions?

A)There is only one operational definition that is possible for each conceptual definition.
B)The specification of operational definitions is one of the creative aspects of the research process.
C)Conceptual definitions are created after operational definitions are determined.
D)Operational definitions and conceptual definitions are the same thing.

Question 7

Dr.Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic.One of her first tasks is to determine which of her participants are narcissistic and which are not.She decides to use the scale created by a colleague,the Mayo scale.Question 1 reads,"I tend not to think about other people as much as I think about myself." Question 2 reads,"I do not have a high opinion of myself." Question 3 reads,"I think other people think I am really special."

Dr.Valencia is concerned whether her measure will really measure narcissism or if it will measure some other related concept.She is concerned about the scale's ________.

A)Operational definition
B)Validity
C)Reliability
D)Convenience

Question 8

Which of the following is NOT true of scatterplots?

A)They can be used to examine interrater reliability.
B)They can be used to examine internal reliability.
C)They should not be used for examining reliability.
D)They are the preferred method for examining all types of reliability.

Question 9

Dr.Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic.One of her first tasks is to determine which of her participants are narcissistic and which are not.She decides to use the scale created by a colleague,the Mayo scale.Question 1 reads,"I tend not to think about other people as much as I think about myself." Question 2 reads,"I do not have a high opinion of myself." Question 3 reads,"I think other people think I am really special."

Before using the measure in her study,Dr.Valencia gives the measure to a group of students on Tuesday.She gives the measure to them again on Thursday.Dr.Valencia is examining the scatterplot of the data she collected on day 1 and day 2.On the scatterplot,she sees that the dots are very close to forming a diagonal line.This indicates which of the following?

A)A strong relationship
B)A nonrelationship
C)A valid finding
D)A negative finding

Question 10

What is the difference between a ratio scale of measurement and an interval scale of measurement?

A)A ratio scale of measurement has a zero value that actually means "nothing" or "the absence of something," but an interval scale does not.
B)An interval scale of measurement is a type of measurement used for categorical measurements,but a ratio scale is used for quantitative measurements.
C)An interval scale has equal intervals,but a ratio scale does not.
D)A ratio scale of measurement cannot be used to compare people's scores,but interval scales can (e.g. ,Phillip is twice as fast).

Question 11

Hosea is studying the relationship between caffeine consumption and problem-solving ability.Which of the following is a categorical way to operationalize caffeine consumption?

A)The number of glasses of soda consumed in a day
B)The number of milligrams of caffeine consumed during the study
C)The frequency of buying coffee drinks
D)Whether the participant drank a soda in the 24 hours prior to the study

Question 12

Todd is also studying the effect of popularity on academic success for his research methods project.He decides to measure popularity by asking each elementary school student to tell him how many friends he or she has.He assumes that more friends means the student is more popular.Which of the following best describes this variable?

A)A ratio scale of measurement
B)A qualitative variable
C)An other-report measure
D)A categorical variable

Question 13

Hosea is studying the relationship between caffeine consumption and problem-solving ability.Which of the following is a quantitative way to operationalize problem-solving ability?

A)The time spent solving a math problem
B)The type of puzzle solved (Sudoku puzzle or a crossword puzzle)
C)Whether participants used insight or trial-and-error techniques to solve the problem
D)The report of a teacher about whether a student is a good or bad problem solver

Question 14

Asking many similar questions when trying to measure a concept is done to:

A)Ensure validity
B)Cancel out measurement error
C)Make sure participants are not lying
D)Allow participants to skip questions

Question 15

Dr.Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic.One of her first tasks is to determine which of her participants are narcissistic and which are not.She decides to use the scale created by a colleague,the Mayo scale.Question 1 reads,"I tend not to think about other people as much as I think about myself." Question 2 reads,"I do not have a high opinion of myself." Question 3 reads,"I think other people think I am really special."

Before using the measure in her study,Dr.Valencia gives the measure to a group of students on Tuesday.She gives the measure to them again on Thursday.She then compares the scores between the two days.This is a test of which of the following?

A)Interrater reliability
B)Internal reliability
C)Test-retest reliability
D)Construct reliability

Question 16

For his research methods class,Felipe plans to watch how teachers treat children in their classrooms who have ADHD.He will evaluate how positively or negatively the children are treated.This is an example of what type of measurement?

A)Self-report measurement
B)Observational measurement
C)Physiological measurement
D)Archival measurement

Question 17

Dr.Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic.One of her first tasks is to determine which of her participants are narcissistic and which are not.She decides to use the scale created by a colleague,the Mayo scale.Question 1 reads,"I tend not to think about other people as much as I think about myself." Question 2 reads,"I do not have a high opinion of myself." Question 3 reads,"I think other people think I am really special."

Dr.Valencia calculates a correlation coefficient (r)to examine the relationship between Question 1 and Question 2 and between Question 1 and Question 3.She finds a correlation coefficient of
R = -0.73 between Questions 1 and 2 and a correlation coefficient of r = 0.74 between Questions 1 and 3.Which of the following is true of her findings?

A)There appears to be good internal reliability in the scale.
B)The correlation between Questions 1 and 2 and Questions 1 and 3 are in the same direction.
C)The correlation between Questions 1 and 2 is much weaker than between Questions 1 and 3.
D)The negative correlation between Question 1 and Question 2 is bad for Dr.Valencia.

Question 18

A correlation-based statistic called ________ is commonly used to determine internal reliability.

A)Cronbach's alpha
B)Kappa
C)A scatterplot
D)Pearson's r

Question 19

Dr.Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic.One of her first tasks is to determine which of her participants are narcissistic and which are not.She decides to use the scale created by a colleague,the Mayo scale.Question 1 reads,"I tend not to think about other people as much as I think about myself." Question 2 reads,"I do not have a high opinion of myself." Question 3 reads,"I think other people think I am really special."

Before using the measure in her study,Dr.Valencia analyzes the data she gets from her students.She looks at the relationship between each of the individual questions.She sees that participants who agree with Question 1 also agree with Question 3 and disagree with Question 2.This is a test of which of the following?

A)Interrater reliability
B)Internal reliability
C)Test-retest reliability
D)Construct reliability

Question 20

Dr.Valencia is considering conducting a study examining whether narcissistic people have poorer social interactions than those who are not narcissistic.One of her first tasks is to determine which of her participants are narcissistic and which are not.She decides to use the scale created by a colleague,the Mayo scale.Question 1 reads,"I tend not to think about other people as much as I think about myself." Question 2 reads,"I do not have a high opinion of myself." Question 3 reads,"I think other people think I am really special."

Dr.Valencia decides to test the internal reliability of her measure.Which of the following results would make her happy?

A)a = 0.10
B)a = -0.03
C)a = 0.95
D)a = -0.98

Question 21

Establishing construct validity would probably be most important for which of the following?

A)A measure of blood pressure
B)A measure of the number of seizures a person has per week
C)A measure of religiosity
D)A measure of obesity

Question 22

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

To test his measure,Dr.Sheffield gives his measure to a group of people in Gamblers Anonymous (GA)and another group of people in Alcoholics Anonymous (AA).He finds that people in the GA group have higher scores on his new measure than people in the AA group.This procedure is known as a:

A)Test-retest paradigm
B)Known-groups paradigm
C)Prediction paradigm
D)Group evaluation paradigm

Question 23

Which of the following NOT is possible?

A)A measure is neither reliable nor valid.
B)A measure is both valid and reliable.
C)A measure is reliable but not valid.
D)A measure is valid but not reliable

Question 24

Another word for discriminant validity is ________ validity.

A)Convergent
B)Asymmetrical
C)Divergent
D)Multiple

Question 25

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

Dr.Sheffield wants to establish the discriminant validity of his pathological gambling measure.He gives his measure and three others to a group of 100 people.Which of the following provides the best evidence for discriminant validity?

A)That his measure is not strongly correlated with a measure of impulsivity
B)That his measure is not strongly correlated with the number of friends people have
C)That his measure is strongly correlated with a measure of alcohol addiction
D)That his measure is strongly correlated with a measure of self esteem

Question 26

In which of the following ways are content and face validity similar?

A)Both involve subjective judgments.
B)Both involve judgments based on participants' opinions.
C)Both are preferred by psychologists as measures of validity.
D)Both are necessary for predictive validity.

Question 27

Why are convergent and discriminant validity often evaluated together?

A)Both terms mean the same thing.
B)Both involve collecting information from a lot of psychological measures of theoretical interest.
C)Both require the use of scatterplots.
D)Both have to be determined by a panel of experts.

Question 28

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

Dr.Sheffield decides to test the criterion validity of his measure.Dr.Sheffield gives his measure to a group of people that includes suspected problem gamblers and nongamblers.Which of the following options below does he also need to do to get evidence for criterion validity?

A)Give the measure to a group of people attending Gamblers Anonymous meetings
B)Two months later,ask the same group of people to report how many times they have gambled recently
C)Ask the participants to give their opinion on whether the measure is valid
D)Give a measure of alcohol addiction to the same group of clients

Question 29

Establishing construct validity is most important for which of the following?

A)A concrete construct
B)A behavior that is directly observable
C)Physical measurements (e.g. ,length)
D)An abstract concept

Question 30

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

Dr.Sheffield gives his measure to his supervisor,who is also an expert in pathological gambling.His supervisor says that his measure appears to test all the components of pathological gambling,including feeling restless when attempting to stop gambling,jeopardizing jobs in order to keep gambling,and using gambling to escape from problems and a bad mood.Given this information,Dr.Sheffield's measure has evidence of which of the following?

A)Content validity
B)Predictive validity
C)Criterion validity
D)Discriminant validity

Question 31

Your friend Dominic is complaining about having to take the Graduate Record Examination (GRE),a test similar to the ACT and SAT that is required to go to graduate school.Your friend Shakendra tells him he shouldn't complain,as statistics show that GRE scores are related to graduate school GPA.Shakendra is speaking to the ________ of the test.

A)Discriminant validity
B)Content validity
C)Convergent validity
D)Criterion validity

Question 32

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

If Dr.Sheffield's measure does not actually measure pathological gambling,his measure is said to lack which of the following?

A)Validity
B)Reliability
C)Conceptualization
D)Operationalization

Question 33

An educational psychologist is testing the discriminant validity of a new measure of numerical learning difficulties.He gives his measure to a group of students along with another measure of verbal learning difficulties,which he predicts should not be strongly related to numerical learning difficulties.Which of the following correlations would the psychologist hope to find in order to establish discriminant validity?

A)r = 1.0
B)r = -1.0
C)r = 0.83
D)r = -0.18

Question 34

What does it mean that "reliability is necessary but not sufficient for validity"?

A)If a measure is reliable,it is also valid.
B)If a measure is valid,it is also reliable.
C)Reliability and validity are unrelated concepts.
D)Reliability and validity are the same concept.

Question 35

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

To test his measure,Dr.Sheffield gives his measure to a group of his clients and at the same time measures how many times they have been gambling in the past month.He predicts that clients who score higher on his measure will also report gambling more times in the past month.This procedure is meant to provide evidence for which of the following?

A)Face validity
B)Content validity
C)Criterion validity
D)Discriminant validity

Question 36

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

Dr.Sheffield has now decided that he wants to test his measure on some university students (who some estimates say have a 6% prevalence rate of compulsive gambling).He has a group of 100 university students complete his measure.He also has them complete two other measures (one that measures addictive behavior in general and one that measures general attitudes toward gambling).He finds that his new measure is positively associated with each of these other measures.This procedure has provided evidence for the ________ of Dr.Sheffield's measure.

A)Content validity
B)Predictive validity
C)Convergent validity
D)Discriminant validity

Question 37

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

Dr.Sheffield has decided to test the discriminant validity of his new measure.He has a group of first-time Gamblers Anonymous (GA)attendants complete his measure and finds that they score higher than a group of people who do not attend the group.Which of the following results would provide evidence for discriminant validity?

A)He finds that the GA attendees score higher on his measure than the non-GA attendees.
B)He finds that the measure of gambling is not correlated with a measure of life satisfaction in the same two groups of people.
C)He finds that more recent GA joiners score higher than veteran GA attendees,who are more recovered.
D)He finds that the measure he used is also associated with people's past diagnoses of pathological gambling.

Question 38

Dr.Sheffield is a clinical psychologist who specializes in treating pathological gambling.Pathological gambling is defined as being unable to resist impulses to gamble.Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition,he creates a new measure of pathological gambling.The measure has 15 questions,and it takes 20 minutes to complete.

To test his measure,Dr.Sheffield gives his measure to a group of people in Gamblers Anonymous (GA)and another group in Alcoholics Anonymous (AA).He finds that people in the GA group have higher scores on his new measure than people in the AA group.Why did Dr.Sheffield do this?

A)To obtain evidence for face validity
B)To obtain evidence for content validity
C)To obtain evidence for convergent validity
D)To obtain evidence for criterion validity

Question 39

In interrogating the construct validity of a measure,which question should a researcher ask?

A)Is there enough evidence that this measure is valid?
B)Do I know that this measure is valid?
C)Does this measure have the right kind of validity?
D)Has an expert said that this measure is valid?

Question 40

Your friend Dominic is complaining about having to take the Graduate Record Examination (GRE),a test similar to the ACT and SAT that is required to go to graduate school.He complains that it doesn't really measure how well he will likely do in graduate school.Dominic is questioning the ________ of the test.

A)Discriminant validity
B)Content validity
C)Convergent validity
D)Criterion validity

Question 41

Lauren,Sarah,and Jennifer are students in Dr.Shaffer's Research Methods class.For a class assignment,they are asked to devise an operational definition for romantic attachment,or love.Lauren decides to ask a group of married couples to report if they are in love with their spouse (1 indicates "Yes,I am in love" and 2 indicates "No,I am not in love").Sarah decides to watch couples sitting on campus and measure how close they sit together,assuming that people who sit closer together are more in love.Jennifer decides to recruit a group of newlyweds and measure the change in heart rate that occurs between the start of the study (when the person is alone)and the moment they see their spouse walk into the room.

For each student described above,indicate whether the operational definition of her variable is self-reported,observational,or physiological.

Question 42

Name the three types of reliability and the four types of validity.

Question 43

Explain the difference between validity and reliability.Explain whether they are related or independent concepts.

Question 44

Dr.Li is interested in creating a measurement of religiosity.According to the dictionary,religiosity is "the quality of being religious;piety;devoutness." He creates a measure comprising 10 statements.People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree,5 = strongly agree).Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."

Dr.Li gives his new measure to a group of 100 adults in order to test the internal validity of his measure.What type of correlation coefficients (r)would Dr.Li expect to find between Statement 1 and Statement 2 and between Statement 1 and Statement 3? What would these correlation coefficients tell Dr.Li about the internal reliability of his measure?

Question 45

Dr.Li is interested in creating a measurement of religiosity.According to the dictionary,religiosity is "the quality of being religious;piety;devoutness." He creates a measure comprising 10 statements.People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree,5 = strongly agree).Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."

Provide a suggestion for how Dr.Li could establish the criterion validity of his measure.

Question 46

Imagine that you are trying to measure people's stress.Provide three quantitative definitions of stress: one that is ordinal,one that is interval,and one that is ratio.

Question 47

Dr.Li is interested in creating a measurement of religiosity.According to the dictionary,religiosity is "the quality of being religious;piety;devoutness." He creates a measure comprising 10 statements.People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree,5 = strongly agree).Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."

Dr.Li gives his new measure to a group of 100 adults in order to test the internal validity of his measure.Draw two scatterplots: one that demonstrates the likely relationship between Statement
1 and Statement 2 and one that demonstrates the likely relationship between Statement 1 and Statement 3.Explain why you have drawn each scatterplot as you have.

Question 48

Dr.Sarolli is planning on measuring literacy (defined as the ability to read and write written text in one's native language).He decides he wants to create a quantitative operational definition.Create an ordinal,interval,and ratio way to measure literacy.

Question 49

Explain why your textbook argues,"In fact,operationalizations are one place where creativity comes into the research process."

Question 50

Lauren,Sarah,and Jennifer are students in Dr.Shaffer's Research Methods class.For a class assignment,they are asked to devise an operational definition for romantic attachment,or love.Lauren decides to ask a group of married couples to report if they are in love with their spouse (1 indicates "Yes,I am in love" and 2 indicates "No,I am not in love").Sarah decides to watch couples sitting on campus and measure how close they sit together,assuming that people who sit closer together are more in love.Jennifer decides to recruit a group of newlyweds and measure the change in heart rate that occurs between the start of the study (when the person is alone)and the moment they see their spouse walk into the room.

For each student described above,indicate whether the operational definition of her variable is categorical or quantitative.If a student's operational variable is categorical,explain whether it could or could not be made quantitative and how you would do that.

Question 51

Explain why many people feel that physiological measurements are the best way to operationalize a variable but why they may not,in fact,be the best.

Question 52

Imagine that you are trying to measure people's stress.Provide an example of measuring stress using a self-report measure,a physiological measure,and a behavioral measure.

Question 53

Given the above situation,explain the difference between criterion validity,convergent validity,and discriminant validity.Why is each one important?

Question 54

Dr.Li is interested in creating a measurement of religiosity.According to the dictionary,religiosity is "the quality of being religious;piety;devoutness." He creates a measure comprising 10 statements.People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree,5 = strongly agree).Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."

Provide a suggestion for how Dr.Li could establish the face validity of his measure and a suggestion for how he could establish the content validity of his measure.

Question 55

Dr.Li is interested in creating a measurement of religiosity.According to the dictionary,religiosity is "the quality of being religious;piety;devoutness." He creates a measure comprising 10 statements.People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree,5 = strongly agree).Some of the statements are below.
Statement 1: "I believe in a religion."
Statement 2: "Part of who I am stems from my religious beliefs."
Statement 3: "I believe that religion is unnecessary."

Name two types of reliability that apply to Dr.Li's measure and how each one could be established,given the above scenario.