And as the journey ends

So as we near the end of this journey, I go back to the original question from that very first day Who am I as a researcher?" And while I still do not know the exact answer and maybe I shouldn't because self identity like qualitative studies are always changing, I do know the following. I really enjoy people. And while there are time that I long to be a cast away on a deserted island bothered only by a lonely iguana or two, for the most part, I need to be around people to hear their stories, share my own and find strength in that community of humankind. I know that as I long as I have an interest in hearing what people have to say and in hearing what they have learned from their experiences, I can probably slide my personality into a qualitative researcher mode of thinking.

I love the stores crate and barrel ,l home goods and the container store. Why is this important"

Because one thing I have noticed through this journey is the amount of stuff that accumulates. Stacks and stacks or articles, books I obsessively buy from Amazon (right Jeanne) pens, paper, highlighters, digital recordings, videos, post-its Anyway all this stuff in the consistent need to organize. There is a lot of organizational skills needed in this kind of research. Those with executive dysfunction disorders will have a hard time, but as long as I can continue to buy color coded folders, baskets, mesh file cabinets and stop cleaning-- I will make it through the organizational piece

I have, as I will admit, a few biases that pop up--my most aggressive one is my own enthusiasm. Perhaps it comes with age. I have raised three children, had a former career, and dispensed with a marriage that was not exactly working, so perhaps my bias for my enthusiasm because I am doing something I want to do, always wanted to do and can now do it shines through all the time. It is not hidden-- That may become a problem if it isn't already, I'll have to see

So who I am as a researcher continues to develop and I think the one final post for right now is that coding wasn't as scary in reality

Validity Reflection

The study that I chose to look at for the validity analysis was Kindle, K. (2010). Vocabulary development during read alouds: examining the instructional sequence. Literacy, Teaching and Learning, 14 (1 &2), 65-88. This was a case study involving four primary grade teachers and how they incorporated effective vocabulary practices into read aloud instruction. I chose this study to analzye because it is very similar to my case study.

The author was clear in stating the research questions which were 1) What specific strategies did the teachers use? 2) How did they link into instructional sequences? 3) How did instructional sequences differ among teachers? After a thorough review of the literature, and examples of three research based models of vocabulary discussion, Kindle went on to cleary articulate the methodology employed in this study. The researcher used systematic observation and proves a rationale for doing so as written in “An Observation Survey of Early Literacy Development” Clay (1993). Kindle took the role of non-participant observer during these observations.

While Kindle was transparent in terms of stating clearly her methodology, she does not clearly articulate her role as a research or does not make any of her research biases clear. In addition, she gives a description of the length of teaching of the four participants (two of whom are described as veteran and two are described as new to the profession); yet, she does not explain how the participants were recruited, or selected (i.e. random, purposeful sampling), and while it is implied that these four teachers were chosen because of years of teaching experience, she does not explicitly state this.

The observation process was clearly described and Kindle by stating that she did not give specific instructions to the teacher to avoid leading them to include particular practices. She uses Merriam (2001) to support this technique.
Kindle provides a clear description of the data analysis. She used Creswell (2008) technique of analyzing transcripts in a recursive or spiral manner, used low inference coding (Carspecken, 1996) to establish categories. Transcripts were re-read and patterns of interaction were matched to the model instructional sequences presented. Inter coder reliability was established through the use of peer debriefing, and the author states that member checks were used to increase validity of her depictions of the data. In addition, Kindle describes that she used the data to form a structured interview protocol and the interview helped triangulate the data to further ensure validity. However, while she mentions the topic domains included in the interview, she does not explicitly state some of the research questions, nor does she use any dialogue from the interview in her transcripts. This would have added to the thick descriptions that she made with her transcripts of instructional sequences. Kindle provided vignettes from the transcripts in the analysis and result section to support conclusions and to compare and contrast the teacher behavior to the model instructional sequences.

Finally, while Kindle goes on to state that no data was obtained measuring student word knowledge or word learning because this was a study interested in teacher behavior, she later draws conclusions about effective vs. ineffective teaching practices with only her examples from the transcript to support. It should have been more clearly stated in the results section that these effective/ineffective practices were specific to these cases and warranted further exploration before they could be generalizable.


Overall, this case study was a good model of a case study for my research. The author is effective in establishing validity and reliability. However, to strengthen the validity, there were several areas where the author could have been more explicit. One, researcher bias nor were the limitations to the study implicitly stated. In addition, explicitly stating that the conclusions that she made regarding effective and ineffective practices were based on these specific cases and warranted further exploration before generalizing seems warranted. Finally, while the interviews served to triangulate the data, the interviews were not incorporated into the discussion section and it is unclear how they were used in the study. The author merely provides a description of the topic domains used to develop the protocol. Example of interview questions or language samples obtained from the interviews should have been woven into the discussion/results section.

Code Red STAT

Code RED -"stat"

Coding is no easy task. How quickly I am seeing that. Got organized, have all those wonderful folders that Dr. Turner suggested. And now, as I begin to sift through the data, (the sifting of my flour for my cookies will wait) I am realizing just how subjective coding can be and I understand the importance of trying to find the common core in the codes and what the sub-codes should be. And then, as I read student responses to questions, I realize the importance of member checking for perception. I have an idea of what some of their vague comments might mean, but is that what they really mean, or is that what my own perceptions think they mean. And when kids say that writing in school is too restrictive- what do they mean? Is it too time restricted, or topic restricted? When they say that there are too many distractions to write, do they mean kid distractions, teacher distractions, environmental distractions and how do those distractions differ? So as I try to group the categories, I see that some of them clearly overlap, but some of them may overlap only because I think they do. I also notice as I read their answers that my questions overlap- I think I could have refined my questions because several of them seem to be asking the same thing although in my mind, when I made them, I was looking for different answers. Their answers however seem to be pointing me in the direction that my perception is different, They are perceiving the two questions as the same, when I did not.

And what do I do (between you and me) with the kids who say they just do not like to write- I like to write, I want them to like to write. I feel disappointed and I wonder if I was a better teacher would they like to write or does it have nothing to do with me? I am so narcissistic.

Oh well, that will be an hour on the couch. Back to coding

"Truthfulness"

For this blog post on validity and reliability or "truthfulness" in qualitative research, I chose to analyze the following journal article:

Scharlach, T. D. (2008). These kids just aren't motivated to read: The influence of preservice teachers' beliefs on their expectations, instruction, and evaluation of struggling readers. Literacy Research and Instruction, 47(3), 158-173. Retrieved from ProQuest Research Library database

This qualitative case study will be included in the literature review of my mini-inquiry project as it relates to one of my research questions: "How, if at all, do the prior beliefs, habits, attitudes, experiences, and background of preservice teachers inform the practice of student/newly certified teachers in regards to early reading instruction?"

I think that the author of this article does a satisfactory job of showing that she both thought about "truthfulness" as she was doing the actual work of research and when she was completing the draft of the text.

To begin with, there is thick description in the piece.  We are given many details about both the participants of the study as well as the methodology involved in the work.  In addition to the narrative, there are nine tables which help to elucidate the findings of the author.  Multiple data sources were used in addition ("questionnaires, autobiographies, interviews, observations, written expectations, and evaluations" [p. 161]) providing for triangulation and structural corroboration of both data sources and methods.  Also listed in the study at multiple points are references to the works of other researchers where similar findings could be located.  These connections from the past works of others in the field provided an additional source of triangulation and structural corroboration.

Although not found in the section on methodology, we are given an entire section of validation and reliability information entitled, "Delimitations and Limitations of the Study" (p. 172).  Here, in addition to the author admitting that any transferability/generalizability to other settings/individuals is modest at best, readers are provided with an account of the researcher's biases.  The author readily admits that these biases could not be separated from this work and "provided the lens through which all of the information was processed" (p. 172).  It was interesting to see that she, just like we did not too long ago, listed her "I"s (Peshkin, 1988) that contributed to the biases affecting and influencing this work.  These "I"s included being a "former classroom teacher, reading resource teacher, graduate student of reading, preservice teacher educator, and researcher of a reading intervention program" (p. 172).  Scharlach also noted that another "I" was that of a "white, middle-class female" (p. 172).

There are some issues however.  Although we are given a full description of how the data analysis was performed, the author used no other coder other than herself, so we have no inter-coder reliability as a result.  Blind coding of the various documents may have taken place but there is no mention of it.  There is also no mention of a codebook and the specific coding scheme used, only noting of the subsequent categories that came out of the author's coding.  Furthermore, there is no talk of a member or participant check of the data.  In regards to other strategies of validation (Creswell, 2007), there is no review by peers of which we are made aware, no negative case analysis or mention of disconfirming evidence, or external audit.

Overall, I think that it is clear that there is a high level of transparency in the research report.  I would say that both the researcher and her results are subsequently credible.  To support my claim here, I will cite Creswell (2007) as he recommends that "qualitative researchers engage in at least two of them [validation strategies] in any given study" (p. 209).  Scharlach actually engages in four out of the eight suggested by Creswell (2007).  I think that I will be able to therefore safely use the information in this article for my project and feel comfortable in its validity, reliability, and truthfulness.

The Truth Do We Really Know What It Is

And I am still questioning who I am as a researcher. And sometimes I wonder if I can really be a successful qualitative researcher. It fits my personality, of that I am sure, but my very set of "I'"s can easily get in the way of validity. So prior to reading all the work on validity, I found that I was (and in many ways still am) intimidated by a lot of the research I read. I have found myself wondering how did they do that, and where did they get the skills to write like that, and how come their articles seems so scholarly and mine so elementary. And then I would have to get up and walk around the house and do laundry and wash dishes as I tried to internalize and process all of this. That being said, I have learned a lot about the hidden dangers in qualitative research. At least in quantitative, the enemy is clear--all those Greek symbols and p's and F's and Z's and D's and Alphas. but in qualitative, it is a little more hidden. A misplaced smile on the face of the researcher, a change in tone when asking a questions, the wording of the question, the relationships between the researcher and the participant, the "God, how much do I want my results to match what I am doing" or even, "Uh-oh, my data does not support my questions, maybe I should change my questions." All of these things and more need to be considered, and perhaps, the memoing and other ideas like that become the proverbial To-do list that I make all the time. I think in addition to my shopping lists and my To-do lists I am going to have to make a validity-list to keep by my side to make sure that they are all being considered.

I read a study Sharmini, (2011) Planning in feedback: Insights from concurrent verbal protocols. And it seemed like a decent study. It had all the buzz words I am looking for writing process, peer feedback recursive nature of writing, revision, and I got very excited, but even though I did notice information in the article that was similar to other research articles I have read, I did notice things that I may not have noticed earlier--before the concentration on validity. For example, the study was a case of three students at the graduate level, but no discussion explaining the reason for the choice of these participants was described. According to Maxwell, (2005) the site selection and participant selection decision should be discussed in order to attain full disclosure. The students were asked questions that related to their writing process and the importance they gave to feedback but I, as the reader, did not know what was asked. The author of the study states, " there was clear indication that engagement with written feedback is not only recursive, but involves planning" (p. 5) but there was no discussion about the definition of engagement or recursiveness or what constitutes planning. This made it difficult for me to determine if a study like this could be applied elsewhere and if it could be applied, the applicable situations was not clear.

In the implications section, the author does provide some follow-up questions such as "What is your main argument here and always link to your stand" (p.5) but without knowing the foundation for this question, I am not sure how to connect this follow up to the research question. The author does present a purpose for the study, " this study was done to gain further understanding on the thought process of writers when they attended to written feedback" (p. 1) but there are no research questions. Now I understand the importance of having research questions. Without the questions, I can not connect the findings and the discussion to the study's purpose. Note however that knowing this doesn't necessarily make doing that any easier for me. I have to engage in a recursive process myself with my own work, constantly going back and reviewing and asking if I am consistently tying my data to my questions. I realize just how easy it is to go off the beaten track and follow the roads that just seem to appear out of nowhere. Being recursive will clearly lead to greater focus. Being recursive has to be added to my validity "to do" list. And speaking of other issues. The author of this study says it is a qualitative work, but I see no mention of her own validity threats. There is some member checking with the participants that help to confirm what the researcher is proposing, but there is no mention of any outside external audits or peer review. In essence, while the information in the article does reflect much of what I have already read in more documented research studies and research based books, I do see the flaws in this study as it relates to the reader's ability to make any conclusions about how this work can be transferred. I understand the difference between a quantitative reliability study with statistical support and I now understand more deeply the methods that qualitative researchers must employ to ground their work as being valid and conclusive.

• 
  

Validity Issues: Wanzek & Vaughn, 2009

While rereading the Maxwell (2005), Chapter 6, “Validity,” what seemed to connect most with a study I am using for my mini-inquiry lit review (Wanzek & Vaughn, 2009; cited below) was the concept of generalization in qualitative research. As Maxell points out, “external generalizability is often not a crucial issue for qualitative studies. Indeed …a qualitative study… may provide an account of a setting or population that is illuminating as an extreme case or ‘ideal type’” (p. 115). Wanzek & Vaughn examined case studies of three students who exhibited extremely low response to reading intervention. More intensive forms of intervention were attempted on the subjects, and it was demonstrated that one of the subjects responded to a more intensified version of the interventions previously tried, while another subject again failed to response to the more intense intervention, and required a whole new approach (whole word reading) to begin responding to the interventions (the third subject dropped out of the study). Clearly, Wanzek and Vaughn’s subjects were extreme cases, and the intense intervention strategies which finally worked on the case study subjects would not be needed for the vast majority of students, so the external generalizability of the study results would be limited. However, as Waznek & Vaughn point out, “teachers of students with disabilities will encounter students with pervasive disabilities in reading and benefit from documented case studies of similar students” (p. 161). As is common to much qualitative research, especially case studies, even though broad external generalizability is not to be expected, studies of extreme cases can still be valuable. The results of these case studies are needed for small populations, for example for students for whom traditional teaching methods and methods developed through large quantitative studies have failed. For this reason, studies like Wanzek and Vaughn, though their application is limited, are still worthwhile. Another validity issue discussed in Wanzek and Vaughn (2009): fidelity of implementation of the interventions. The study states that “six tutors …were hired and trained by the research team…” (p. 154). The tutors received 15 hours of training in instructional techniques, lesson planning, progress monitoring, and group management techniques. The researchers reviewed the tutors’ lesson plans and ran practice sessions with them. Throughout the interventions, the tutors were observed each week and given feedback on their implementation, and monthly fidelity checklists were completed for each tutor. Each tutor received a rating for the quality of their implementation, and ratings for all tutors were consistently high. Knowing that the tutors were trained and monitored so well gave me confidence that the interventions were implemented with fidelity, and consistently for each subject. Wanzek, J., & Vaughn, S. (2009). Students demonstrating persistent low response to reading intervention: three case studies. Learning Disabilities Research & Practice, 24(3), 151-163.

Where I stand today....

Dear members of Qualitatively Speaking, Okay, so in my self reflection, one of my goals was to better use the blog as a tool to reflect on my growth as a researcher. In an attempt to do so, I decided I would post the answer to the final two questions on the mid-term self assessment on the blog. I think that the answers to these questiosn will change greatly as the semester nears the end and we have all had the experience of completing our mini-inquiry project. I would love to get your feeback and thoughts on my repsonses and to see where you are all at, too. :) Christy How can I use qualitative approaches to uncover answers to important questions? Qualitative research allows me to uncover answers to important questions in a way in which my personal experience, background, philosophies, etc. can be used as a lens to interpret data. I believe that qualitative allows for this type of interpretation and discourse in a way in which quantitative does not; however, I am not able to clearly articulate this point at this time. Who am I as a reseracher? I am still struggling to answer this question. Perhaps because I do not yet see myself as a researcher rather a researcher in progress. However, maybe that is what all researchers are? Researchers in progress because I would imagine that being engaged in the process of inquiry forces one to continuosly involve and grow. Right now, I am keenly aware that I, as a researcher, postion myself as an advocate of students with learning disabilities. I feel it is important, when possible, to include the voice or lens of the participant whenever possible so as not to have my lens or view dominate. For instance, in my case study, I plan to interview the teacher after taking field notes to clarify. I supsect that this will be a very useful and meaningful activity in addtion to serving as a source of data triangulation. Note: I just noticed an interesting phenomenon. I typed the answers to these questions in word and uploaded to drop box. I then decided to publish this on the blog. However, I am not able to figure out how to cut and copy text from word into the text field and so I ended up looking back at the original word document and then answering the questions again. This time, the answers to my questions were more thorough, reflective, and quite frankly better written! I can't wait to see how these answers change at the end of the sememster! :) Disclaimer: Since I am challenged in terms of knowing how to upload from Word, there maybe typos in this post. I apologize!