How Context Clues are Assessed on STAAR

During a recent meeting with Gayla Wiggins and Lead4Ward, Gayla said, "Context clues are not assessed the way they are used." It blew my mind. I met with a group of friends to tease apart the ideas. Essentially, we decided that we are teaching context clues as a skill rather than as a process.

Using context is a metacognitive process, not a set of steps you go through to find out what a word means. Telling kids to look in the sentence to find the clues won't always work. Telling kids to look at the sentences before and after the word is used won't always work either. As a matter of fact, "the chances of using a single context to derive a word's meaning are slim (only about 15 percent of the time) and doing so often requires and extensive amount of inference" (Trainer notes, Reading to Learn Academy, Vocabulary, slide 19). That shocked me.

But it does make sense. Nuanced text hardly ever gives a single piece of evidence. Yet nuanced text does give multiple pieces of overarching context if you use the whole text to make meaning. To use my favorite example, we can't teach kids to answer vocabulary questions as if it were a Seek and Find on a Chili's children's menu. If kids are reading the question, then going to the text to find the word, reading "around" the word, that's not context. That's isolation. No amount of circling or highlighting is going to help the child understand the word. Nuanced text won't have enough information to help the reader understand the meaning of the word. Besides, reading isn't about understanding one word. Reading is about how all the words coalesce to deliver the author's message. Context clues aren't about understanding the words. Context clues are about how you use the words to understand the point of the text. Understanding one word is not -and hopefully will never be -the point of authentic writing or reading.

The following examples come from the 8th Grade 2017 Released Reading STAAR Exam.

Here's an example. The question asks students to look for a word found in paragraph 2. The evidence listed in the multiple choice items is also from paragraph 2.

But look at how much evidence is peppered THROUGHOUT the text supports the meaning of that word. Reading the ENTIRE text is the context. Not one sentence. Not one paragraph. By teaching kids to only look at a portion of the text without consideration of the meaning of the whole text is a bizarre test taking strategy to handcuff kids to certain misconceptions about context clues as well as how we are really supposed to use context in actual life. Again, the focus of reading this passage is not to understand what scant means. It's about something more important. 

And here's another example. Just because the word is in paragraph 5 doesn't limit the evidence and context to paragraph five. Understanding the WHOLE passage brings the reader meaning about how the word is used and connected to what the author is saying with this word. In this case, a secret strategy for success.

If I had the time, and thought you would read it, I'd record a think aloud about how each piece of highlighting helps me understand the words and the author's message. But, I don't have the time. And you probably didn't read all the way to the end anyway. 

Choosing Vocabulary for Deep Thematic Links

Choosing Words

Fluency Applied to Writing: Handwriting and Typing

Part of the reading academies this summer addressed fluency. In our sessions, some of us started talking about the connection between fluency and handwriting. I started looking for some guidance and found some interesting research.

When kids compose, their handwriting and typing speed can inhibit their ability to transfer their ideas onto paper. Here are some important principles I learned:

• Find out how fast kids can copy information. There is a limit to how fast people can copy information. The fastest speed that they can copy information legibly should follow these rates: 
• 1st - 5 words per minute
• 2nd - 6
• 3rd - 7
• 4th - 8
• 5th - 10
• 6th - 12
• 7th - 14
• 8th - 16 (Amundson, 1995). 
• The faster you can copy, the more room you have in working memory to process thought. So kids who have trouble with automaticity and fluency in handwriting or typing can negatively impact the content of their writing. 
• Kids in 4th grade typically compose using handwriting at 4-5 words per minute. By the time they are in the 9th grade, they compose using handwriting at 9 words per minute (Graham, 1990). Most adults compose at about 19 words per minute (Karat, Halverson, Horn, Karat ,1999).
• If kids can type at least 10 words per minute, that seems to match handwriting speed pretty closely. That means that if they type slower than that, typing actually makes it harder to compose. I think that should mean that kids should reach a benchmark in typing speed before being allowed the privilege of using the computer to compose. (Excluding assistive technology use.) If you can't type as fast as you write, you're going to be frustrated. 
• Handwriting ability and keyboarding ability are NOT associated. In other words, if you have good handwriting it doesn't mean you can type well or not. BUT, kids should learn the basics of handwriting before learning to type. A concentrated focus on typing then, should probably begin after kids learn to write in cursive - my opinion. 
• When composing, most adults who type can produce between 10 and 18 words per minute (Foulds, 1980). Most 5th and 6th graders who can type well usually compose at about 6 words per minute (Graham, Harris, MacArthur, and Schwartz, 1991).   So 10 words per minute minimum seems like a good benchmark for deciding how fast kids should be able to type before begin asked to compose on the computer. Again, my opinion.
• Most research suggests that effective use of software requires a baseline of about 20 words per minute. 
• Some suggest that typing speed should be 2 to 3 times faster than handwriting speed. 
• Boys tend to type slower than girls, but almost catch up by 8th grade (Honaker, 2003).
• Most of the businesses that I looked at online require at least 30 words per minute. The ability to type 40 words per minute is considered an average typing speed. Professional typists have averages above 65 to 75 words per minute. Some people think most of the population doesn't have the finger dexterity to type above 50 words per minutes. So, looks like we should have kids typing between 30-40 words per minute by the 8th grade. 
• The technology apps for 6, 7, and 8th grade don't mention a specific speed. 
• The fastest anyone ever typed is 216 words per minute! 

Link to Resource with charts

Link to Standards document that might be useful in curriculum development and integration of these ideas.

Review of Literature 

Another review for handwriting speed

An assessment you can use

References:

Amundson, S.J. (1995). Evaluation tool of children's handwriting. Home, AL: O. T. Kids.

Foulds, R. A. (1980). Communication rates for nonspeech expression as a function of manual tasks and linguistic constraints. Proceedings of the International Conference on Rehabilitation Engineering. Toronto.

Graham, S. (1990). The role of production factors in learning disabled students' compositions. Journal of Educational Psychology. 82, 781-791.

Graham, S., Harris, R. K., MacArthur, C., & Schwartz, S. (1991). Writing and writing instruction for students with learning disabilities: Review of a research program. Learning Disability Quarterly. 14, 89-114.

Graham, S., Beringer, V., Weintraub, N., & Schafer, W. (1998). Development of handwriting speed and legibility in grade 1-9. Journal of Educational Research, 92(1), 42-52.

Karat, C., Halverson, C., Horn, D., Karat, J. (1990). Patterns of entry and correction in large vocabulary continuous speech recognition systems. Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '99). New York, NY: ACM. pp. 568-575. doi: 10.1145/302979.303160.

Inference Hooey: Part III Addressing Process for Building a Mental Model

The second thing people do when they make inferences is building a mental model. Especially when they don't have one to begin with. And that's a real life skill, ya'all. I have to do stuff all the time that I know nothing about. Like learn how to turn on the dad gum television. Or build one of those boxes so I can look at the eclipse.  But that's another set of stories.

Comprehension Purpose Questions: 

To me, this implies two things: Either WE set a purpose for them. Or THEY set a purpose for THEMSELVES. Both are essential. Most sane people don't do things they don't have reasons for. Reading isn't any different. Setting a purpose before you read allows you to, well, have a purpose. In addition, it allows you to:

• have a basis from which to examine the ideas, 
• think actively, 
• have a standard from which to monitor comprehension, 
• and establish what content you must review to reach understanding (Center for the Improvement of Early Reading Achievement, 2001; Collins-Block, Rodgers & Johnson, 2004; Coyne, Chard, Zipoli, & Ruby, 2007; Duke & Pearson, 2002; Dycha, 2012; Vaughn & Linan-Thompson, 2004). 

Set by the Teacher:  The reading academies suggested that teachers develop and pose Comprehension Purpose Questions (CPQ's)  before students read. The example in the academy came from an article on the Shasta Dam. "How does the Shasta Dam help people in California?" and "How is the building of the Shasta Dam an example of humans overcoming nature?" were two good examples of CPQ's. "In what state is the Shasta Dam?" and "When was the Shasta Dam built? were non examples. 

Set by the Student: The reading academy did not address a strategy for setting their own purposes for reading, but there are many resources for teaching students how to do this work. 

Here are some ideas: 

1. Anticipation Guides (Befre/After)

2. Say Something Chart (During) 

3. SQP3R (Before, During, After)

4. List/Group/Label (Before)

5. Probable Passage (Before/After)

6. Coding Text (During) 

Caution: Don't forget that the point here is NOT about the strategies or posing questions to interrogate the reader. The point here is that getting your mind right about what you are about to read is a process that you go through so that you can comprehend. It's a mental model that good readers use to be successful. 

Types of Inferences: Kids also need to have a mental model about the types of inferences that might confront them as they make sense of text. Beers lines these out for us along with suggestions (2003). Perhaps modeling each of these instances would be a good starting place for mini-lessons. 

1. recognize the antecedents for pronouns
2. figure out the meaning of unknown words from context clues
3. figure out the grammatical function of an unknown word
4. understand intonation of characters' words
5. identify characters' beliefs, personalities, and motivations
6. understand characters' relationships to one another
7. provide details about the setting
8. provide explanations for events or ideas that are presented in the text 
9. offer details for events or their own explanations of the events presented in the text
10. understand the author's view of the world
11. recognize the author's biases
12. relate what is happening in the text to their own knowledge of the world
13. offer conclusions from facts presented in the text (Beers, 2003, p. 65). 

Use Text Structure: 

Authors are purposeful in selecting text structures and genres that will most effectively convey their message and purpose and voice. This reminds me of learning how to play canasta. And most recently of teaching my grandson to play Old Maid. When grandma taught me to play canasta, we started playing with an open hand while I learned the rules. That way she could advise me about the choices evident from the cards in my hand. Text structure is how we learn to play the game with narrative and expository texts. What we have in our "hand" determines how we play the game. If you don't know the possibilities and strategies that structure the game, you probably aren't going to win. Or you can throw a fit and tip the game table like my grandson did when he drew the old maid from the deck. That too. I think sometimes that's what kids do, they just refuse to play/read because they think they can't win. Fortunately, the game doesn't have to be over when you draw the Old Maid. 

In the TEA academies, their recommendations for narratives included: discussing the relationships among characters, setting, and events and linking relationships to the larger theme. Beers goes a bit further and suggests that we should have an anchor chart of the types of inferences and specifically name them as we model and discuss the connections. 

Recommendations for informational tests included using specific structures commonly used and learning key words associated with each of these structures. 

For both genre types, TEA suggested the use of graphic organizers. I don't think you need any help in finding those. 

Caution: Again, let's be clear. The point is not to identify text structures, but to use them to help make inferences because the provide a mental model for the process we use as readers to make meaning. 

Beers, K. (2003). When kids can't read: What teachers can do. Portsmouth, NH: Heinemann. 

Collins-Block, C., Rodgers, L., & Johnson, R. (2004). Comprehension process instruction: Creating reading success in grades K-3. New York, NY: Guilford Press.

Coyne, M. Chard, D., Zipoli, R. & Ruby, M. (2007). Effective strategies for teaching comprehension. In M. Coyne, E. Mame'enui, & D. Carnine (Eds.) Effective strategies that accommodate diverse learners (pp. 80-109). Upper Saddle River, NJ: Pearson.

Duke, N. K. & Pearson, P. D. (2002). Effective practices for developing reading comprehension. In A Farstrup and S. Samuels (Eds.), What research has to say about reading instruction. (pp. 205-242). Newark, DE: International Reading Association.

Dycha, D. (2012). Comprehension, Grades K-3. In M. Hougen & S. Smartt (Eds.), Fundamentals of literacy instruction and assessment, Pre-K-6. Baltimore, MD: Brookes.

Vaughn, S., & Linan-Thompson, S. (2004). Research-based methods of reading instruction: Grades K-3. Alexandria, VA: ASCD.

STAAR Scoring for Writing: Conversation with TEA

I just got off the phone with TEA. I have tried to replicate what I learned with fidelity to TEA's message; however, I realize that there are probably ways that I state my understanding that TEA would NOT officially endorse. 

(For background understanding of why I called TEA: I have been getting a lot of questions about how the essays were scored, what we can do to improve scores, and the possibility of offering a scoring training.)

1. TEA is aware of inconsistencies in scoring and are addressing them as well as they can with the “constructs of the contract.” There is a rescoring policy in place if you feel there are papers that need to be reconsidered.
2. They HAVE NOT changed their tenets to value formulaic writing. TEA’s advice about authentic products is still in place.
3. Paragraphing is great in the real world. Real world writing is not constrained by genre or 26 lines. They do NOT look at paragraphing AT ALL for STAAR.
4. The biggest problems are with essays that try to follow these patterns:
1. 4 paragraph essays with no ideas fully developed; they repeat or say nothing. There’s not enough room in 26 lines for 4 paragraphs.
2. 3 paragraph essays with two body paragraphs that contain examples. The examples don’t say much. (In other words, the ideas are not developed.)
5. Perfunctory transitions like first, next, finally do nothing for the paper. Transitions should link the ideas between paragraphs and connect back to the thesis.
6. Kids are not planning or prewriting in their booklets. Only about 30 percent are doing anything at all. And most of those just start writing and then recopy what they wrote onto the answer document.
1. TEA recommends that we focus more on the PLANNING element of prewriting.
2. TEA recommends that we help kids have the courage to cross out stuff/delete information that does not go directly with the thesis. (I thought that was a nice way to say that.)
7. Kids are still getting stuck in the box/stimulus. The 4th graders last year really had a problem with that. (We need to help them find the prompt – the writing charge.)
8. Biggest problems between 1’s and 2’s were: no thesis and weak conventions
9. Biggest problems between 2’s and 3’s was development of ideas and organization. Kids who used anecdotes included too much information that didn’t connect back to the thesis. There was too much repetition.
10. Biggest problems between 3’s and 4’s was still development of ideas and organization. Kids who could use historical information and cultural facts accurately seemed to do the best. Using knowledge from other disciplines seemed to work well. Organizational structures really pushed the papers up to a 4. They try to include these examples in the scoring guides. (I would add that this is not demonstrate causality. Writing about history or culture is not going to get you a 4. It’s just that kids who are doing better in writing have more knowledge and schema to draw from.)
11. There is always a lot of talk about addressing the prompt. TEA is responsive to kids and allows the kids to approach the prompt however they choose. For example, the fourth grade essay about meeting someone they have never met? They allowed kids to write about fictional characters, JJ Watt, and even allowed a kid to write about meeting a grandmother for lunch as long as they talked about that person AND why they wanted to meet them.
12. There is a technical digest published that gives us guidance on how raters are trained. Look at this link, starting on page 16: http://tea.texas.gov/Student_Testing_and_Accountability/Testing/Student_Assessment_Overview/Technical_Digest_2015-2016 The new one for last year should be published in February.
13. They use the scoring guides as the anchor papers for the training sets. We don’t have those for last year’s testing yet, but we have them for previous years. 
14. They were going to provide us with an online training guide that ETS uses for training raters. Because of the changes in TEA organization and budget, this probably will NOT be released.
15. The new organizational structure will take place after September 1^st. There will no longer be a distinct division between Content folks and Assessment folks. Staff will be working on BOTH content and assessment.

Scheduling Components for the ELAR Classroom

When I attended the Lead4Ward ELAR Academy recently, I posted a picture about my learning about how to schedule all of the components into a 45 minute classroom. This is a terrible struggle for many teachers, and certainly one of the most frequently asked questions when I lead staff development. "How do I fit it all in?" 

Gayla Wiggins recommended 2 components per week. She said that we need to also be flexible because we are a "responsive" discipline. 

Here are some more of my notes based on her recommendations: 

1. It might feel disjointed at first. You will have a problem with the "flow" at first, but you need to know that it is ok to shift from one thing to another. 

2. The key is balance in getting in all the things that students need. 

3. It's easier to plan with a theme, but you don't have to. Be sure to get LOTS OF TEXTS into their reading. Be careful when you are planning that you don't become so focused in finding stuff that matches your theme that you forget the WHY behind your choice. The purpose is to teach kids the reading and writing processes they will need for success. 

4. Shared reading time gets shorter throughout the year and more and more time is reserved for independent reading. 

5. Notice the balance of time between reading and writing. 

Inference Hooey: Part III Addressing Process to Fill the Gaps

I have to be honest with  you. I am writing about inference to learn about it. Two sets of experiences led me to this exploration. When I was at the Reading Academy training in Austin, they introduced us to some new ideas about inference. I'd never ever ever heard that before. I wasn't really sure what all of these types of inferences were, but a first glance made me think of processes.

Then I attended the first day of the lead4ward academy and and heard a controversial statement: on STAAR, we don't assess context, inference, summary, or connections the ways readers use them to read.  I'd never ever ever thought that before. After I picked my jaw off the floor, I realized, context, inference, summary, and connections themselves are processes that readers use to comprehend text.

After using these "tools to know," there are watch to show our thinking about the text.

These things are identifying theme, topic, main idea, writing a summary, and discussions tone and mood. All of these require the reader to use the first row concepts to process the thoughts. All of these require the reader to make inferences. 

The $100,000 Question: So what does that look like in a lesson? 

Begin with QAR:

Students need to understand the kind of thinking we are asking them to do. Do they know when they need to infer? The QAR strategy is probably the best example. It's a strategy that helps the reader understand the relationship between the question (the cognitive task) and the right answer. I think that's the place to begin. Then you can begin teaching the processes described in the TEA slides. 

Fill the Gaps: Ask Questions and Consider the Author's Intentions

Ask Questions about What's Missing: Do kids realize that stuff is missing in a text? I'm thinking about the Goldilocks example. Why did Goldilocks break Baby Bear's chair? The text never says that Goldilocks does it on purpose. It was an accident. That's missing. The chart below shows something else that is missing - it says that Goldilocks is not a baby. The text called her a little girl, but it never called her a baby. I'd have to say that's missing in the text too. The text never says that Goldilocks weighs more than Baby Bear either. Those are all reasoned inferences based on things not present in the text. 

Ask Questions to Monitor Your Comprehension:

So the effective reader with inference superpowers needs to continually monitor their comprehension by asking questions to make sense of the text. I taught it like this: 

I had kids draw the pictures in their journals while I modeled the process on the chapter book I read aloud to the class. Then I gave them all little Popsicle stop signs and had them partner read. The partner would stop and ask the reader questions. They'd discuss and reread or trade tasks if everything was clicking along. 

Ask Questions about  the Author's Intentions: 

Why did the author include or exclude certain information? 

Why did the author choose to write it in that way? 

Why did the author choose the genre?

What is the author trying to accomplish? 

The reading academy included a lesson asking questions about the author's intentions. You can download the participant materials here. Look for the Comprehension section, Handout 4. You can also press cntl and the f key simultaneously. Then you can search for Chicken Sunday to quickly navigate to the right page. 

Caution: The questions do a good job of connecting to the author's intent, but I don't think it goes far enough. Here's an example: The author says that Stewart and Winston were her "brothers by a solemn ceremony" and that their gramma was her gramma. Why does she describe them this way?  I think following up with a question about why the author chose to include that detail about the ceremony and why she chose to use a colloquial term for grandmother would be a better question. Then you could talk about how those choices in craft helped the author deliver the central message more effectively. 

Ask Carefully Planned Questions during Frequent Read Alouds: 

To plan a read aloud for this purpose, divide the text into four chunks. After each chunk, pick 3-4 vocabulary words to develop and discuss that the writer chose that give the writing power and beauty and are key to establishing understanding, particularly connotation. Then create a question that you can pose to the class about filling in the gaps of the reading. Write words and the question on a bookmark or sticky note. 

Fill the Gaps: Connect Background Knowledge to Text Evidence: 

This image sums it up pretty well about how to make this work in the classroom. 

This link from ASCD gives the entire chapter of Overcoming Textbook Fatigue. It lists some fantastic lesson ideas and approaches to add to your tool-belt. 

The next two blogs will address Process to Build a Mental Model and Process to Make the Text Cohere.