Exam 12: Systematic Observational Methods
What are the three dimensions or facets of "naturalness?"
The concept of "naturalness" can be understood through various lenses, depending on the context in which it is being considered. However, in the context of environmental philosophy and ecology, "naturalness" often refers to the degree to which an ecosystem or a landscape is unaffected by human influence. In this regard, the three dimensions or facets of "naturalness" typically include:
1. **Ecological Integrity**: This dimension refers to the wholeness and functioning of an ecosystem. An ecosystem with high ecological integrity has a complete range of species, trophic levels, and ecological processes that are characteristic of that ecosystem type. It operates without significant disruption from human activities, allowing for natural evolutionary processes to occur. Ecological integrity is often measured by the presence of native species, the absence of invasive species, and the functioning of natural ecological processes such as nutrient cycling, natural disturbance regimes, and predator-prey dynamics.
2. **Wildness**: Wildness pertains to the degree of human control or dominance over a landscape or ecosystem. An area with a high level of wildness would be one where natural processes and forces such as fire, predation, and climate dictate the structure and function of the ecosystem, rather than human management or intervention. Wildness is often associated with remoteness and inaccessibility, and it can be seen as a measure of the absence of human manipulation or presence.
3. **Historical Fidelity**: This facet of naturalness is concerned with the extent to which an ecosystem or landscape resembles its condition prior to significant human alteration. Historical fidelity takes into account the species composition, structure, and function of an ecosystem as it would have been before industrialization or other major human impacts. It emphasizes the importance of maintaining or restoring ecosystems to their historical states, or at least to states that closely resemble those conditions.
These three dimensions of naturalness are not mutually exclusive and often overlap. For instance, an area with high ecological integrity is likely to also score high in wildness and may have a high degree of historical fidelity if it has been relatively untouched by modern human activities. Conservationists and environmental managers may use these facets to assess the state of naturalness in an area and to guide their efforts in preserving or restoring natural environments.
Researchers assessed the interrater reliability of their pilot study coding scheme, and found that its reliability was lacking. What are some ways they could enhance the reliability of their category system or coding scheme? (circle all that apply)
B,D
What is Cohen's kappa? How is agreement or reliability between coders determined?
Cohen's kappa is a statistical measure used to assess the agreement or reliability between two or more coders when coding categorical data. It takes into account the agreement that would be expected by chance and provides a more robust measure of agreement than simple percent agreement.
To determine the agreement or reliability between coders using Cohen's kappa, the following steps are typically followed:
1. Select a sample of data that needs to be coded by multiple coders.
2. Have each coder independently code the same set of data.
3. Calculate the observed agreement between the coders, which is the proportion of times they agreed on the coding.
4. Calculate the expected agreement by chance, which takes into account the distribution of codes and the marginal totals for each category.
5. Use these values to calculate Cohen's kappa coefficient, which ranges from -1 to 1. A value of 1 indicates perfect agreement, 0 indicates agreement equivalent to chance, and negative values indicate agreement worse than chance.
Overall, Cohen's kappa provides a more nuanced understanding of agreement between coders by accounting for chance agreement and is widely used in fields such as psychology, medicine, and social sciences to assess inter-rater reliability.
You are a research assistant in a developmental psychology program, and tasked with observing children's expression of feelings and emotions, on the playground, to other children and adults. How intensive or extensive would your categorization system be? In your response, (a) explain and justify why you would use a primarily intensive or extensive category system, and (b) provide an example for how five units of observation would be categorized in your system (i.e., provide 5 examples of behavior or other units of observation that would be noted in your categorization system - are those observations hierarchically or laterally organized?).
Three researchers wanted to determine the interrater reliability of their field observations regarding aggressiveness of dog encounters at the dog park. Specifically, for each dog-pair, the researchers noted the breeds (or suspected breeds) of each dog, and how aggressive the dogs' first interaction was (1 = not at all aggressive; 5 = moderately aggressive, 10 = they got into a fight). To determine the interrater reliability of their aggressiveness observations, which of the following techniques would be most appropriate and yield the highest (reliable) reliability value?
Which of the following would be an example of a unit of observation?
Maurice and Nora are planning an observational study, and trying to decide on how involved they will be in the observational setting. To avoid influencing people's responses by being present, they decide to use the experience sampling method (ESM) to obtain systematic observations. Their choice of involvement in the naturalistic setting would be considered:
Which of the following features of a study are aspects of "naturalness?"
You do not want to "go native" in an upcoming observational study because you fear your in-person presence will unduly influence people's daily behaviors and internal states. Describe two methods of partial participatory observation and two methods of non-participatory observation. Provide an example for each type of partial- and non-participatory observation.
Can a research laboratory (where participants would typically participate in psychological experiments) be an appropriate setting for naturalistic observation? In what situations might this be applicable? Why would a researcher want to use a laboratory as the preferred setting to observe "natural" behavior? What would be an example of two studies where the only difference between them is that one utilizes naturalistic observation to assess the dependent measure, and the other uses an "unnatural" (e.g., written self-report) measure to assess the same dependent construct? Does naturalistic observation primarily involve indirect assessments of the constructs we are trying to measure - why or why not? Could there be a situation where the participant is knowingly interacting with the experimenter in a laboratory setting, but the data being collected is still considered "natural behavior?"
For some reason, you are fascinated with dog parks, and want to conduct a systematic observational study of the dynamics of people's behavior while at the dog park. What are the important factors to consider when designing your study? For example, given that you want the study to be very naturalistic, you would want to ensure that the behaviors being exhibited are an existing part of people's repertoire, that the setting is not perceived by park goers to be artificially constructed for the purposes of research, and that any "treatments" are such that park goers would have experienced these discrete events with or without the presence of a researcher. Besides the degree of "naturalness," what are three of the factors that one must consider when designing a naturalistic observational study? In your response, be sure to (a) describe each of the three factors; (b) explain the different considerations for each factor, and the benefits and drawbacks to these considerations; and (c) provide an example for how each factor would manifest in the context of observation of the dog park (i.e., what would your decision regarding each factor look like had you actually conducted the observational study?).
Answer template:
Factors or dimensions to consider:
a. observer involvement in the natural setting
b. how extensive or intensive will the coding scheme be
c. will they use qualitative, quantitative, or both types of coding
d. how will they classify the unit of observation.
Which of the following would be considered non-participatory observation? (circle all that apply)
Bales' 12-category interaction process analysis (IPA) would be considered a more _____(a)_____ system. Caldwell's 65-category APPROACH coding scheme would be considered a more _____(b)_____ system.
Dalton and Elanor are preparing an initial coding scheme for an upcoming field observation of people's behaviors in shopping malls. When developing their coding scheme, all of the following statements are true, EXCEPT:
Using an _____(a)_____ system of categorizing ensures (at least to some degree) that we will note most, if not all, potentially relevant event or behaviors. _____(b)_____ categorization systems have fewer details recorded for any given event or behavior.
You are interested in the phenomenon of "basking in reflected glory," where people tend to affiliate themselves with successful others (e.g., wearing sports paraphernalia), without any personal connection to those successful others, in the hope that some of this success or others' positive attributions will "reflect onto them." Describe one way in which you could investigate this phenomenon with an extremely high degree of naturalness. Then, describe one way in which you could investigate this phenomenon with a very low degree of naturalness. How would they differ? In your response, be sure to indicate (a) the different features of a study that determine the extent to which it is considered "natural," (b) how those features would be manifested in studies of high and low naturalness, and (c) the specific ways in which you would alter those features for your specific examples to obtain high and low naturalness.
You are interested in how teenagers think about decision making, and want to use naturalistic observation to assess the frequency of different thoughts, and valence of those thoughts, that a person experiences when asked to respond to a given stimuli (e.g., when asked whether he or she would try a new type of illicit drug). How could you do it? Could you use facial expressions in some way? What about eye movement or other unobtrusive physiological movements or responses? How else could you observe cognition (albeit indirectly) as unobtrusively and as "naturally" as possible? What might your coding scheme look like for these behavioral observations?
Describe what is meant by the intensity-extensity distinction in coding systems. In which instances might a researcher adopt one approach over the other, and with what consequences?
Naturalistic observational studies offer which of the following benefits?
Filters
- Essay(0)
- Multiple Choice(0)
- Short Answer(0)
- True False(0)
- Matching(0)