After a closer consideration of details and hidden costs, the overall price tag for our longitudinal study now stands at approximately $550,000. This jump (and it is a jump: six months ago we hesitated at the thought of crossing the six-digit barrier) coincides with an orientational shift: We’re taking much of the focus off our specific courses and teachers and moving it more toward our students’ socioeconomic backgrounds and experience with reading and writing. In other words, after making a distinct shift from academic to institutional research, we’re now moving toward what might be called community-based research. Of particular interest is New Hampshire (i.e. the state)’s approach toward education and the very real problems with literacy our seemingly privileged and homogeneous body of students appears to be facing.

As a result, the working title we’ve used in the pilot (Four Years of Writing) is probably heading to that big title repository in the sky. First, we might be looking at a six-year study, covering a year in high school and a year after college (and that’s just the synchronic bit… in reality, our questioning will reach clear back into our students’ first literate experiences in childhood). Second, we’re probably going to start emphasizing the term literacy.

I’m known for my good titles (if nothing else). But since we’re applying to the Spencer Foundation, we can’t be too crazy. Title suggestions are welcome.

Meanwhile, I now find myself in charge of a half-million-dollar study. And this is merely my grunt work, completely independent of my coursework and dissertation. I think I’m nuts.

(And here comes a rant)

Meanwhile, I’ve made a few decisions about how I, the director of the whole thing, am going to frame the study from now on. What strikes me, as people express concerns about the study’s “lack of consistency” (because we have unique conversations with each student-interviewee instead of set questions) or “lack of reliability” is how limited their sights are. The questions they want to see answered—the answers that could result from standardized questioning methodologies and coded, reliable taxonomies or response—are easy questions. Facile. The type of stuff that you shouldn’t be spending this kind of money on because you could easily get it from a five-dollar survey. What this “cult of consistency” (which exerts strong influence even in composition studies, as I learn anew with each passing 4Cs) continues to push is exactly the type of meaning-making that has left us dissatisfied as we’ve looked at previous studies.

My attitude at this point is to say, “Okay. We’ll work in all of the ‘valid/reliable’ (pick your term here, since they’re conflated) study elements you want. In fact, we’ll do that quickly and easily. And the results we give you will be predictable, uninteresting, and will utilize only a small percentage of the usefulness of this study. Meanwhile, we’ll spend our spare time in the background engaged in rich dialogue on questions you’re not interested in, questions that can’t be pre-planned or scaled for reliability because they’re generated within the process of the interview. Guess which part of the study will say more to the university, the state, composition studies, and our students.”

I’m sorry, but when it comes to assessment inquiry, questioning methods deeply rooted in the humanistic tradition of “hermeneutic dialectic” (Guba and Lincoln’s term) win out every time. Why? Because they don’t sacrifice reality for consistency, or deep learning for packaged results, or dialogue for outcomes.

If the study progresses and we start to see patterns, great. But I’m not going to narrow my questioning to make students’ experiences more commodifiable. That’s false consistency. Besides, in my mind, one student saying something is just as important as all of them saying it. Although one student’s words might not affect policy as much as the words of many, their story is real and it deserves to be heard.

Some questions worth asking: In what ways is standardization of research valuable? In what ways is it harmful? And why is uniformity equated with “fairness”?

(End of rant)