Week 5 Discussion Case 1- The Hawthorne Effect

In qualitative research there are several data collection techniques that are utilized to amass data, they are listed as:

Interviews,
Questionnaires and Surveys,
Observations,
Focus Groups,
Ethnographies, Oral History, and Case Studies and
Documents and Records.

ORDER NOW FOR CUSTOMIZED SOLUTION PAPERS

The two collection data errors that I chose from the reanalysis of the “Hawthorne Effect” researched by Levitt and List were:

Bias in the study and,
Loss of documentation and records.

Bias

Modification of the study was instituted by the subjects because they “knew” that they were being observed. Qualitative research is a form of social inquiry. It is vital to understand that qualitative research is not a single type of social inquiry. The variations in approaches in qualitative study requires

Interpretive – qualitative research focuses on understanding the way people interpret and make sense of their experiences and the world in which they live
Naturalistic – qualitative research studies social phenomena in their natural settings

Levitt and List needed to take into account when they reanalyzed the results that subjects are “likely to modify behavior when they are aware that they are part of an experiment, and this would be difficult to quantify” (McDavid et al., 2013) so many years later.

Loss of Documentation

The historical value of the study was lost and qualitative research involve “the use and study of a variety of empirical materials” (Langbein, 2012); case studies, personal experiences, introspective, life stories, interviews, observational, historical, interactional and visual texts. Due to the inconsistency, no conclusions could be drawn as to what took place and why. The data became skewed, at that point.

Reanalyzing the data typically involves gathering empirical materials using some form of observation or interviewing method; and this could not be completed due to the fact that the study was completed in the 1920s.

Works Cited

Langbein, L. (2012). Public program evaluation: A statistical guide (2nd ed.). Armonk, NY: ME

Sharpe. Chapter 7, “Designing Useful Surveys for Evaluation” (pp. 209–238).

Levitt, S., & List, J. (2009). Was there really a Hawthorne effect at the Hawthorne plant? An

analysis of the original illumination experiments. Retrieved from

http://www.nber.org/papers/w15016.pdf.

McDavid, J. C., Huse, I., & Hawthorn, L. R. L. (2013). Program evaluation and performance

measurement: An introduction to practice (2nd ed.). Thousand Oaks, CA: Sage.

Chapter 4, “Measurement for Program Evaluation and Performance Monitoring” (pp. 145–185)

RE: Week 5 Discussion – Case 2

COLLAPSE

Top of Form

Geddes (1990) argument for the pitfalls of selecting cases based on dependent variables in program evaluation is relevant because it may result in sample selection bias and false inferences. When selecting cases based on the dependent variable, the researcher will collect data from the groups of which they choose; thus, collecting half of the data. Therefore, the results will be different than they would be if one studied a sample that included a control group. Real scientist will search for evidence that goes against their theory to test their hypothesis. Geddes mentioned faulty inferences that can occur when the results show a true correlation; however, the outcome is based on bits and pieces of information. The outcome of a study or program evaluation may cause one to underestimate the effect of the independent variable(s). Although no research study is 100% true, researchers should aim to get as close to the truth as possible. Evidence-based research is vital to policies and programs, which is why it is important to refrain from sample selection bias. Selection bias is a validity issue for research studies. In a quantitative study, researchers strive to achieve generalizable results. However, if internal validity is not achieved, external validity will not be possible.

Collecting data through sample selection bias may not be viewed as feasible and dependable in social science research, but there are instances when selecting cases based on the dependent variable may be acceptable. One case where selection based on the dependent variable is acceptable is quantitative case-control designs (Forgues, 2012). For example, if the study is about the effects of mentor programs conducted in minority communities, the sample selection would consist of mentor programs in minority communities. Another example, if the evaluation team seeks to focus on a mentoring program for post-release juveniles, their sample would include post-release juveniles rather than post-release adults. This would be purposive sampling which can be thought of us a validity issue of selection bias. However, the approach may be the most appropriate to gain the knowledge that is most beneficial to the study or evaluation.

References

Forgues, B. (2012). Sampling on the dependent variable is not always that bad: Quantitative case-control designs for strategic organizational research. Strategic Organization, 10(3), 269-275. doi:10.1177/1476127012452820

Geddes, B. (1990). How the cases you choose affect the answers you get: Selection bias in comparative politics. Political Analysis, 2(1), 131–150. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.372.5896&rep=rep1&type=pdf

REPLY QUOTE EMAIL AUTHOR

Bottom of Form