STAAR Scoring for Writing: Conversation with TEA

I just got off the phone with TEA. I have tried to replicate what I learned with fidelity to TEA's message; however, I realize that there are probably ways that I state my understanding that TEA would NOT officially endorse. 

(For background understanding of why I called TEA: I have been getting a lot of questions about how the essays were scored, what we can do to improve scores, and the possibility of offering a scoring training.)

1. TEA is aware of inconsistencies in scoring and are addressing them as well as they can with the “constructs of the contract.” There is a rescoring policy in place if you feel there are papers that need to be reconsidered.
2. They HAVE NOT changed their tenets to value formulaic writing. TEA’s advice about authentic products is still in place.
3. Paragraphing is great in the real world. Real world writing is not constrained by genre or 26 lines. They do NOT look at paragraphing AT ALL for STAAR.
4. The biggest problems are with essays that try to follow these patterns:
1. 4 paragraph essays with no ideas fully developed; they repeat or say nothing. There’s not enough room in 26 lines for 4 paragraphs.
2. 3 paragraph essays with two body paragraphs that contain examples. The examples don’t say much. (In other words, the ideas are not developed.)
5. Perfunctory transitions like first, next, finally do nothing for the paper. Transitions should link the ideas between paragraphs and connect back to the thesis.
6. Kids are not planning or prewriting in their booklets. Only about 30 percent are doing anything at all. And most of those just start writing and then recopy what they wrote onto the answer document.
1. TEA recommends that we focus more on the PLANNING element of prewriting.
2. TEA recommends that we help kids have the courage to cross out stuff/delete information that does not go directly with the thesis. (I thought that was a nice way to say that.)
7. Kids are still getting stuck in the box/stimulus. The 4th graders last year really had a problem with that. (We need to help them find the prompt – the writing charge.)
8. Biggest problems between 1’s and 2’s were: no thesis and weak conventions
9. Biggest problems between 2’s and 3’s was development of ideas and organization. Kids who used anecdotes included too much information that didn’t connect back to the thesis. There was too much repetition.
10. Biggest problems between 3’s and 4’s was still development of ideas and organization. Kids who could use historical information and cultural facts accurately seemed to do the best. Using knowledge from other disciplines seemed to work well. Organizational structures really pushed the papers up to a 4. They try to include these examples in the scoring guides. (I would add that this is not demonstrate causality. Writing about history or culture is not going to get you a 4. It’s just that kids who are doing better in writing have more knowledge and schema to draw from.)
11. There is always a lot of talk about addressing the prompt. TEA is responsive to kids and allows the kids to approach the prompt however they choose. For example, the fourth grade essay about meeting someone they have never met? They allowed kids to write about fictional characters, JJ Watt, and even allowed a kid to write about meeting a grandmother for lunch as long as they talked about that person AND why they wanted to meet them.
12. There is a technical digest published that gives us guidance on how raters are trained. Look at this link, starting on page 16: http://tea.texas.gov/Student_Testing_and_Accountability/Testing/Student_Assessment_Overview/Technical_Digest_2015-2016 The new one for last year should be published in February.
13. They use the scoring guides as the anchor papers for the training sets. We don’t have those for last year’s testing yet, but we have them for previous years. 
14. They were going to provide us with an online training guide that ETS uses for training raters. Because of the changes in TEA organization and budget, this probably will NOT be released.
15. The new organizational structure will take place after September 1^st. There will no longer be a distinct division between Content folks and Assessment folks. Staff will be working on BOTH content and assessment.