Deck 12: Evaluation Research

Question 1

Melbin "Night as Frontier" study (1978) used field experiments to test their hypotheses.

Answer 1

True

Question 2

The author's reputation and previous articles should have no bearing on how you assess the findings of a journal article.

Answer 2

False

Question 3

Surveys have limited use for understanding context and mechanism.

Answer 3

True

Question 4

Applied research reports should be written with a non-academic audience in mind.

Question 5

Despite conscientious use of sampling, reliable coding procedures, and the use of multiple measures, Melbin's "Night as Frontier" study has limited generalizability because of self-selection of subjects.

Question 6

Use the following to answer questions 16-18:
In each set, match a concept from Group A to a definition or example from Group B.
Sections of a Research Report
Group A
1. Discussion section
2. Appendixes
3. Findings section
4. Executive summary

Group B
a. Contains compelling foci for additional research
b. Interpretation of data with respect to study hypotheses
c. Back matter, such as copies of the research instrument
d. In an applied report, summarizes study's main findings
e. Presentation of pertinent data and data displays

Question 7

Experimental designs are strongest for testing causal hypotheses.

Question 8

In a research proposal, data displays should be presented in the conclusions section.

Question 9

Use the following to answer questions 16-18:
In each set, match a concept from Group A to a definition or example from Group B.
Sections of a Research Proposal
Group A
1. Ethics statement
2. Statement of limitations
3. Statement of research problem
4. Literature review
5. Methodological plan

Group B
a. Explanation of how research builds on what has already been reported
b. A careful list of anticipated costs
c. How you will identify and respond to problems associated with using human subjects
d. Introductory clarification of what the research is about
e. A review of weaknesses in the project and plans for minimizing their consequences
f. Discussion of how data will be collected, given opportunities and constraints

Question 10

To write a successful report, a researcher should respect the complexity of the task and should not expect to write a polished draft in a linear fashion.

Question 11

A good literature review does all of the following except:

A) Indentifies gaps in the literature
B) Provides ideas for how to design new research
C) Liberally uses direct quotes from other articles
D) Uses up to date and classic research
E) Is selective in articles included

Question 12

When critiquing other research for methodological weaknesses, attend to measurement approach and sampling strategy.

Question 13

A diligent writer will construct a reverse outline before beginning work on a research proposal or report.

Question 14

The introductory statement of the research problem clarifies what it is the researcher is interested in studying.

Question 15

Research proposals should not include data from a pilot study, because these data are likely to be unrepresentative.

Question 16

Use the following to answer questions 16-18:
In each set, match a concept from Group A to a definition or example from Group B.
Strengths and weaknesses of research designs
Note: Answers from Group B may be used more than once.
Group A
1. Strongest for testing nomothetic causal hypotheses
2. Strongest for testing ideographic causal explanations
3. Extremely time consuming
4. Most generalizable
5. Strongest for measuring subjective meanings
6. Able to measure characteristics of relatively large samples
7. Low generalizability because of unrepresentative sampling
8. Low generalizability because of artificiality of setting

Group B
a. Experimental designs
b. Surveys
c. Qualitative methods

Question 17

The foundation for a good research proposal is a good literature review.

Question 18

In applied research report, an executive summary is included at the end of the report.

Question 19

When critiquing the methods of published research, which two elements are most important to assess?

A) Independent and dependent variables
B) Measurement approach and sampling strategy
C) Date of publication and length of article
D) Model tested and variables used
E) Hypothesis and Conclusion

Question 20

The budget should not be included in your research proposal.

Question 21

A research proposal would not include:

A) A literature review
B) A methodological plan
C) A statement of limitations
D) A summary of findings
E) Pilot study results

Question 22

In a research proposal, in which section do you identify how you will respond to working with human subjects in a responsible manner?

A) The introductory statement
B) The literature review
C) The methodological plan
D) The ethics statement
E) The statement of limitations

Question 23

An applied report is written for:

A) Students
B) Manuals
C) Guides
D) Non-academic audience
E) Bureaucrats

Question 24

Experienced writers suggest all of the following except:

A) Leave final editing to the last-minute
B) Don't expect to write a polished draft in a linear fashion
C) Leave enough time for dead ends and revisions
D) Ask others to review and react to what you have written
E) Write as you go along

Question 25

Which of the following is typical of an applied research report but rarely used in a conventional report?

A) An appendix
B) A bibliography
C) An executive summary
D) A findings section
E) A literature review

Question 26

In a research proposal, in which section do you detail how you will collect data?

A) The introductory statement
B) The literature review
C) The methodological plan
D) The ethics statement
E) The statement of limitations

Question 27

In reviewing a research article, which is not a criteria used to assess its quality?

A) What is the basic question being posed?
B) Is the theoretical approach appropriate?
C) Is the literature review adequate?
D) Does the research design suit the question?
E) Do you agree with the hypothesis?

Question 28

When considering its relative merits as a research method, surveys have:

A) High measurement validity, high generalizability, and reasonable causal validity
B) Low measurement validity, but high generalizability and causal validity
C) High measurement validity, but low generalizability and causal validity
D) Low measurement validity and generalizability, but high causal validity
E) High measurement validity, low generalizability, and almost no causal validity

Question 29

Which of the following research methods is best for establishing internal validity?

A) Experiments
B) Surveys
C) Participant observation
D) Focus groups
E) All of the above

Question 30

The ability to use preexisting, reliable measures is a major strength of which type of research method?

A) Experiments
B) Surveys
C) Participant observation
D) Focus groups
E) Field experiments

Question 31

In what section of a research report should you address weaknesses in your own research design?

A) Introductory statement
B) Methodology section
C) Findings section
D) Discussion section
E) Conclusion section

Question 32

According to Chambliss and Schutt, writing is:

A) Fun!
B) Revising!
C) Arduous!
D) Both a and b
E) All of the above

Question 33

When assessing previous research, which should you not consider:

A) Review process for the journal
B) Author's reputation
C) Author's past research
D) Reputation of the university where the author works
E) How research was funded

Question 34

In a research report, in which section do you present pertinent data displays?

A) The introductory statement
B) The findings section
C) The discussion section
D) The conclusion section
E) The appendixes

Question 35

Compared to experiments and surveys, participant observation has:

A) Higher generalizability and measurement validity
B) Higher measurement validity and causal validity
C) Higher generalizability but lower causal validity
D) Lower generalizability and causal validity
E) Lower measurement validity and higher generalizability

Question 36

In a research proposal, in which section do you clarify what you are interested in studying?

A) The introductory statement
B) The literature review
C) The methodological plan
D) The ethics statement
E) The statement of limitations

Question 37

Which of the following research methods is best at creating and refining complex concepts?

A) Surveys
B) Field experiments
C) Qualitative methods
D) Secondary data analysis
E) Statistics

Question 38

In a research report, in which section do you summarize findings and draw implications for the theoretical framework used?

A) The introductory statement
B) The findings section
C) The discussion section
D) The conclusion section
E) The appendixes

Question 39

In a research report, in which section do you include copies of the instruments used?

A) The introductory statement
B) The findings section
C) The discussion section
D) The conclusion section
E) The appendixes

Question 40

Two major drawbacks of qualitative methods are:

A) They have low generalizability and are time consuming.
B) They have low generalizability and rely on the truthfulness of the respondent.
C) They are time consuming and may miss nuances of a respondent's feelings.
D) The rely on the truthfulness of the respondent and may miss nuances of a respondent's feelings.
E) The have low generalizability and rarely specify causal mechanisms.

Question 41

When reading a research article, how do you assess its quality? Provide detailed guidelines for determining if a research article is well written, theoretically sound, and methodologically sound.

Question 42

Which of the following is a guideline for writing a successful report?

A) Respect the complexity of the task
B) Leave enough time for dead ends
C) Write as fast as you comfortably can
D) Ask anyone you trust for reactions
E) All of the above

Question 43

Why is the organization of a research report important? Explain three techniques proposed by Chambliss to help organize a research report. Which of these would be most effective for helping you organize a report? Explain.
Answer Key

Question 44

What is the difference between an abstract and a conclusion in terms of content?

Question 45

Outline the sections of a research report. Explain the purpose of each section.

Question 46

After you have completed a draft of a research report, which of the following techniques will help you improve your organization?

A) Construct a reverse outline
B) Seek constructive criticism from others
C) Use a word processing program to facilitate reorganizing
D) Leave enough time to revise
E) All of the above

Question 47

How does your audience affect the way that you will write up a research report?

Question 48

Which of the following is an example of back matter in an applied research report?

A) An appendix
B) A conclusion section
C) An executive summary
D) A findings section
E) A literature review

Question 49

__________ are the strongest for testing nomothetic causal hypotheses.

A) Experimental designs
B) Interview designs
C) Web-based designs
D) Non-experimental designs
E) None of the above

Question 50

Describe the strengths and weaknesses of experimental designs, surveys, and qualitative methods. Provide concrete examples of research questions that are appropriate for each type of research method.

Question 51

Outline the sections of a research proposal. Explain the purpose of each section.

Question 52

What is the purpose of a literature review in a proposal and a research report?

Question 53

When critiquing prior research, which of the following should not be considered?

A) Are there any grammatical errors or improper sentence structures?
B) How was the report reviewed prior to the publication or release?
C) What is the author's reputation?
D) Who funded and sponsored the research?
E) All of the above should be considered

Question 54

Which of the following is a guideline to writing a conclusions section?

A) Distinguish clearly your own opinion of prior research from conclusions of the authors of the articles you have reviewed
B) Make it clear when your own approach is based on the theoretical framework you are using rather than on the results of prior research
C) Acknowledge the potential limitations of any empirical research project
D) Both a and b
E) All of the above

Question 55

A successful report must be well organized and clearly written. List some of the principles formulated by Booth, Columb, and Williams (1995) in order to write a report.

Question 56

Explain five different ways to facilitate writing a long research report, as suggested by Chambliss. Which of these would be the most effective for helping you write a report? Explain.

Question 57

Which of the following is a not good idea when writing a research report?

A) Draw on the research proposal and on project notes
B) Use a word processing program
C) Write a polished draft in a linear fashion
D) Seek constructive criticism from others.
E) Outline a report before writing it.

Question 58

How does an applied research report differ from a journal article? Explain the differences, while describing the content of report sections that differ between these two types of writing.

Question 59

Which of the following is not a part of the research proposal?

A) Introductory statement
B) Literature review
C) Conclusions
D) Budget
E) Ethics statement