"Truthfulness"

For this blog post on validity and reliability or "truthfulness" in qualitative research, I chose to analyze the following journal article:

Scharlach, T. D. (2008). These kids just aren't motivated to read: The influence of preservice teachers' beliefs on their expectations, instruction, and evaluation of struggling readers. Literacy Research and Instruction, 47(3), 158-173. Retrieved from ProQuest Research Library database

This qualitative case study will be included in the literature review of my mini-inquiry project as it relates to one of my research questions: "How, if at all, do the prior beliefs, habits, attitudes, experiences, and background of preservice teachers inform the practice of student/newly certified teachers in regards to early reading instruction?"

I think that the author of this article does a satisfactory job of showing that she both thought about "truthfulness" as she was doing the actual work of research and when she was completing the draft of the text.

To begin with, there is thick description in the piece.  We are given many details about both the participants of the study as well as the methodology involved in the work.  In addition to the narrative, there are nine tables which help to elucidate the findings of the author.  Multiple data sources were used in addition ("questionnaires, autobiographies, interviews, observations, written expectations, and evaluations" [p. 161]) providing for triangulation and structural corroboration of both data sources and methods.  Also listed in the study at multiple points are references to the works of other researchers where similar findings could be located.  These connections from the past works of others in the field provided an additional source of triangulation and structural corroboration.

Although not found in the section on methodology, we are given an entire section of validation and reliability information entitled, "Delimitations and Limitations of the Study" (p. 172).  Here, in addition to the author admitting that any transferability/generalizability to other settings/individuals is modest at best, readers are provided with an account of the researcher's biases.  The author readily admits that these biases could not be separated from this work and "provided the lens through which all of the information was processed" (p. 172).  It was interesting to see that she, just like we did not too long ago, listed her "I"s (Peshkin, 1988) that contributed to the biases affecting and influencing this work.  These "I"s included being a "former classroom teacher, reading resource teacher, graduate student of reading, preservice teacher educator, and researcher of a reading intervention program" (p. 172).  Scharlach also noted that another "I" was that of a "white, middle-class female" (p. 172).

There are some issues however.  Although we are given a full description of how the data analysis was performed, the author used no other coder other than herself, so we have no inter-coder reliability as a result.  Blind coding of the various documents may have taken place but there is no mention of it.  There is also no mention of a codebook and the specific coding scheme used, only noting of the subsequent categories that came out of the author's coding.  Furthermore, there is no talk of a member or participant check of the data.  In regards to other strategies of validation (Creswell, 2007), there is no review by peers of which we are made aware, no negative case analysis or mention of disconfirming evidence, or external audit.

Overall, I think that it is clear that there is a high level of transparency in the research report.  I would say that both the researcher and her results are subsequently credible.  To support my claim here, I will cite Creswell (2007) as he recommends that "qualitative researchers engage in at least two of them [validation strategies] in any given study" (p. 209).  Scharlach actually engages in four out of the eight suggested by Creswell (2007).  I think that I will be able to therefore safely use the information in this article for my project and feel comfortable in its validity, reliability, and truthfulness.