Get the Scoop on Literacy Assessments

Aug 31

About the Episode:

Rachel Beiswanger serves as a K-12 reading specialist. She works directly with students as an interventionist and also leads literacy initiatives to implement evidence-based practices at her school. She holds a master of education in reading and has been an educator for 12 years. Rachel is passionate about helping educators bridge research to practice and believes that prevention is the key to ending the literacy crisis.

In this episode we chat about:

What a universal screener is and what specific ways it gives us an overall view
How to move from a universal screener to more of a diagnostic assessment
The possible red flags that come up in an assessment
How to support specific literacy skills

Episode Transcript

Hanna:
Hello, everyone. Welcome to another episode of My Literacy Space Podcast. Today, I'm talking with my friend, Rachel, all about assessment. So welcome, Rachel, to the podcast.

Rachel:
Thank you so much.

Hanna:
I'm really excited to have you because I think this is a bit of a hot topic as we've made the switch and learned more about the science of reading, and we're switching to structured literacy. What on earth do we start to assess? We've been used to benchmark assessments, whether people were using, I don't know, Fountas and Pinnell or things like that. And that is just, now that we know that those three queuing systems that we were looking at through that lens are no longer supported by the science of reading, so when we talk about assessments, what is important? And maybe let's talk about why it's important to really be looking at the type of assessment we're doing.

Rachel:
Sure. Through a science of reading lens, when you're thinking about assessment, it's really important to be thinking about essential early literacy skills. So, when you shift out of that level text world, and that type of instruction with three queuing, you also have to shift to evidence-based assessment. And so, you want to be thinking about the big fives, so phonemic awareness, phonics, fluency, vocabulary, and comprehension.

Rachel:
So when you're assessing, you want to be thinking of those five areas, those are really what a strong universal screener will be assessing.

Hanna:
Okay. So when you talk about a universal screener versus maybe a more diagnostic, tell us about maybe an example of a universal screener and what specific ways that that gives us that overall view, and then how do we fine tune it to figure out... Eventually, we're going to talk about how do we form then some targeted skill groups. So, we understand what kinds of those, the big five, we have to address those. So how does that guide then our reading and spelling instruction once we look at that universal screener?

Rachel:
Sure. The universal screener is going to have measures that will give you information about those essential early literacy skills. And so, as a classroom teacher, when you have that universal screening data, you get that picture of a student's overall reading health. So it's essential first and foremost that you do give all the measures. It's a common practice for some schools or some classrooms to just give one measure because they really just want that piece of information, because maybe they're sensing that fluency is the issue, so that they give the fluency measure, whatever.

Rachel:
But when you have that overall picture, you can then look at your data and you can say, "Okay, I've got 20 kids and this percentage of those kids are struggling in phonemic awareness. And this percentage of kids are struggling in letter sound correspondences," and so on and so forth.

Rachel:
Then you can start to get a feel of what your instruction should mostly be focusing on and you can make some general groupings for those small skill-based groups. Typically, the measures that assess... And by the way, I'm thinking of, you asked for an example as well of a screener, so I suggest Acadience Reading K-6. It's what I was trained in, so I guess I'm biased, but it just really aligns with the research really well. And it's just makes sense.

Rachel:
So the measures are for sound fluency and phoneme segmentation fluency, those are both phonemic awareness measures. And then you have nonsense word fluency, which is an indicator of letter sound correspondences and learning the alphabetic principles. So can they blend those sounds together to rate whole word? And then that beyond nonsense word fluency and into the middle of first grade and above to grade six, you have oral reading fluency or ORF, and that particular measure is an indicator of reading comprehension, word attack skills, so advanced phonic skills, fluency. And what [inaudible 00:04:30] saying? I think I said them all. Reading comprehension, accuracy and fluency are the big ones.

Rachel:
So once you have your data, you can start to form some general groupings to put all your assessments in, these are the students who are still needing letter sounds. So we're going to probably have a group for that, that's targeting that skill. These are the students who are reading accurately, but they're not fluent. So they're reading slowly, they're below level in oral reading fluency. So we're going to have a group devoted to fluency. These students are not reading accurately, so obviously they're going to be reading slowly and their comprehension's probably going to be affected. And if they're not reading accurately, that's a decoding issue. And so, this group's going to be focused on decoding.

Rachel:
And then you're probably going to have a group of kids who are on grade level and really just need that grade level instruction.

Hanna:
Great.

Rachel:
And for the group that is not accurate and it's a decoding issue, the screening is not likely. You could look at the errors on the screener to get a general sense of what kids are missing, but you're likely going to need to do a decoding diagnostic for...

Hanna:
So when we talked a minute ago, the universal screener and now we're focusing on, now we've got this overall score from Acadience, so that was the one that you had mentioned. Now we look at that and we are looking at, these are some maybe deficit areas or some lower scores in certain students. Now, when do we know to move from that universal to more of a diagnostic assessment? How do I know that piece?

Rachel:
Yeah. So anytime you have a student who fault, who scores below the benchmark, that's a red... So something needs to change for them. And if it's not obvious, for the measured nonsense word fluency, sometimes it can be obvious like they're scoring really well in their letter sounds, they're accurate on those, but they're just struggling with blending. So they need to work on blending. You don't need to do a diagnostic for that. But for the kids who are below level on oral reading fluency, there's so many possibilities for those kids. But rule of thumb, if they're below benchmark, give a diagnostic, and the diagnostic should assess whatever specific measure they're below the benchmark on. If it's phoneme segmentation fluency, you might give them a phonemic awareness diagnostic because that's an indicator, phonemic awareness?

Hanna:
Right. Okay, let's talk about fluency. Let's give a little bit of a pretend we have a student in the room with us. And what is the sort of overall screener looking for in fluency? Like you were talking about some of those pieces, it could be multiple areas. So let's look at that ORF piece. What could be possible red flags that come up in that part of the assessment?

Rachel:
So for fluency, the assessment, the measure is words correct per minute. So students are reading that passage for a minute, so there's a score that they would get that they would be at benchmark, however many words it is per minute. So that's how fluency is assessed is that measure. And that is a really well-researched and affirmed by research way to assess fluency and comprehension.

Hanna:
So we do this, is it one minute, you said?

Rachel:
Yeah.

Hanna:
In that specific one? Okay. So once you look at that one minute, how would I know? What are the common questions? Because when we're moving from that level, leveled reading assessment into these really specific, now we know that they scored below that benchmark for fluency.

Hanna:
So now I'm going to look at what would be the next kind of assessment for that. You'd suggest that I'm going to be like, "Okay, now I need to look at some really specific targeted skills so that I know even within fluency, we could be having a group or two, that might be a specific concept that they have to be beefing us."

Rachel:
Sure. So fluency is a little bit easier, at least in my experience. So this is anecdotal. That's a little [inaudible 00:09:03] to pretend kid.

Hanna:
Yeah.

Rachel:
So it's a little easier to target in terms of instruction because yes, it includes things like rate and prosody and accuracy. But really, we're not even talking about accuracy. We're talking about the ability to fluently read those words. So if it is fluency, you may be working on things like scooping phrases. And when you're giving the universal screener, on Acadience, there's a section that you would check mark reading behaviors. So you are maybe making notes about reading every word robotically.

Hanna:
Or something like that.

Rachel:
When you can like, okay, I'm going to want to model for this group. Model fluent reading, practice reading and phrases, reading in scoops and that kind of thing.

Hanna:
So the universal screener, again, if you're looking at the Acadience, some people have said, "I don't have 30 minutes of time per student." And I think that you and I have talked outside of the podcast, but what length of time would that universal screener take per students so that you really get a good glimpse of where you can begin to group or begin to dive deeper into how can I support specific target literacy skills?

Rachel:
Yeah. So it definitely depends on the student, but I would say Acadience has an average time that it takes for student, but I don't know it off the top of my head, but I would say that it probably takes between anywhere between five and 15 minutes. And a lot of schools, what they'll do is they'll set up a schedule where all the adults and support people come in and people who've been trained to give the assessment. Reading specialists will come in and everybody will do the benchmark assessments. So it's less on the classroom teacher. I like to give them because I'm going to be the one instructing them. And so, I like to have that insider perspective on, I like to do it, but also sometimes, that's not realistic in larger school settings.

Hanna:
For sure.

Rachel:
Yeah.

Hanna:
And then how often would that screener be done throughout the year? So now we've done it maybe first month of school. And then when do we repeat to reform groups or rejig what we are talking about?

Rachel:
So the inner practice is beginning middle and end of year, and are segueing into progress monitoring. Progress monitoring is absolutely where we're going.

Hanna:
Yeah.

Rachel:
So a mentor of mine, and I stand by her statement, shared with me that progress monitoring every single one of your students is really hard to accomplish. It takes a lot of time, especially if you're using a new assessment, and you've just started all this new signs of reading instruction. Whoa, this is a lot. So she recommended to start with your lowest group. And even if that's too much to start with your lowest kid and get a system going, but you want to progress monitor frequently so that you have data points that you can look at and say, "Okay, my instruction is working or it's not working."

Hanna:
Right. Real quick.

Rachel:
Oh.

Hanna:
Yeah.

Rachel:
Because if you're only progress monitoring once a month, it's not going to really tell you much. You won't know until December that you need to make a change and that's not quick enough.

Rachel:
So, the other piece of that is using the data that you gain from the universal screener to create a goal for that student. And so, you'll create that goal. And then you'll progress monitor with a measure that supports that goal. So for example, if you have a first-grade student who was below benchmark in nonsense word fluency, then you would be progress monitoring them with that nonsense word fluency measure, and just that. So you wouldn't progress monitor with every measure from the universal screener, you would just use that NWF to track their progress.

Hanna:
Yeah. You know what, and I think that those are the pieces that do take up a lot of time. So what's one way that you've made keeping track of the data, keeping track of who needs quick assessment? What are some quick tips that you might be able to give to classroom teachers or other educators? I know homeschool families even have the same question. They want to know how... Okay. We've talked about how often, how do we keep that data organized or what are some tips you could have on that?

Rachel:
Yeah. So the good news is the measures all take one minute only in terms of how long the student is reading something back to you or performing one of the tasks. And then, it's really important to create a schedule for yourself. So sometimes, a common practice is for teachers to progress monitor every Friday, but it's more efficient if you're picking one kid a day or two kids a day to progress monitor during a transition time, during kids are eating breakfast, and your student finishes early, you're pulling that kid over and you're doing their progress monitoring, so definitely having a schedule. And I keep my progress monitoring booklets in my face, so I don't forget.

Rachel:
Because we want to spend time on instruction and it's like, "Oh, I got to do that. I have to do that." But it is so important. And I always think you're not losing too much instructional time because they're still practicing a skill that's important when they're doing the assessment.

Hanna:
So on average, is it usually what I've seen in when I took a peak at the Acadience too, was that it had the benchmarks? Were they percentages, if I remember correctly?

Rachel:
Yep.

Hanna:
I'm I have one on my bulletin order by the computer. What was the sort of average percentage where you're looking at if they're above or below?

Rachel:
It depends on the measure that you're looking at and the there's the Reading Composite Score and that weights all the scores of each measure evenly. And so, the composite score is really your true overall summary of how the student is doing. But that is a number, not a percentage.

Rachel:
And numbers are used for nonsense word fluency and phoneme segmentation fluency because it's just like how many words and how many sounds, but percentages are used for accuracy. You're going to percent accuracy score in oral reading fluency.

Hanna:
Right. Okay. So let's talk about then some of the other pieces. So we've talked a little bit about the fluency component. Let's talk about one of the other major components, vocabulary. How do we assess vocabulary? That's trickier because everybody's coming to the table with different background information, different world experience. So what are the measures that we look at when we're assessing vocabulary?

Rachel:
So this is the million dollar question, because as you said, everybody has a different experience, everybody has a different level of background knowledge and connection to different things happening in the world and in their lives. And so, vocabulary is one of those skills that is, it's not finite decoding, and can't really be measured in that way. I do know that Acadience is currently developing a measure for vocabulary, but it's not currently on their universal screener.

Rachel:
So to me as a teacher, what I pull from that is a focus on instruction and trying to make my vocabulary instruction definitely a clear piece of my [inaudible 00:17:51] block and something that I'm doing, focusing on high utility words for kids and just bringing the focus to instruction. So I don't have a great answer for that one.

Hanna:
No, but I think that was another question that was pretty popular in preparing for the podcast. People were saying like, "Well, how do you test vocabulary?"

Rachel:
Yeah.

Hanna:
Yeah. There's not very many ways that we can do that. I guess one of the ways that I've done it or I've seen other teachers do it as well, is within a specific unit. Like you said, coming back to that instruction, coming back to pick 10 keywords that you really need them to know, and you can do those pre and post tests, or you coming into a rocks and minerals unit and you say, "These are the top 10 words that you need to know. Do you know them? What do they mean?

Rachel:
Yeah.

Hanna:
Okay. Over the time, these are words that I want you to know." And those would maybe be more like tier two, tier three words because they're very content-specific. So we can target those in all of our read-alouds and really picking out some of those important adjectives or parts of speech that we can explicitly teach. What are some ways that you have, would suggest teaching vocabulary or then coming into a new unit to-

Rachel:
Sure.

Hanna:
Because I think that's another question people are saying like, "Okay, how do I know, in this unit, what are the most important words that, going forward, they relate to other concepts?" I always come back to rocks and minerals because I know that's a unit in grade three.

Rachel:
Yeah.

Hanna:
There's a lot here, like that one, it's not something that a lot of kids come with having a lot of knowledge about rocks and minerals. Right?

Rachel:
Yeah.

Hanna:
That's not something unless your family is really into hiking or [inaudible 00:19:45] like that. So what are ways that we can do that post assessed, talking about a concept, like a unit and then assess it?

Rachel:
Yeah.

Hanna:
How do you do that?

Rachel:
So what you said reminded me of a recommendation that Nancy Hennessy has in her book, The Reading Comprehension Blueprint, about assessing vocabulary. And by the way, if you're looking for ways to assess those language comprehension strands of the route, that's the book for that.

Hanna:
Okay.

Rachel:
But she does suggest along the lines with what you said, if you want to assess vocabulary for a unit, then you could do something... I forget what it's called, and I forget who did the research around it. But basically, you pick out your words. So this is you've already picked the words that you're going to be teaching, and you want to know what kids know then and what kids don't. You have kids rate these words.

Rachel:
So one, I've never seen this word before in my life, two, I've seen this word before, but I probably wouldn't be able to use it in a conversation or a sentence, and three, I know this word really well. I use it in daily conversation. That kind of thing. That's just I guess a general way to know what you really need to explicitly teach. And then maybe you can look at your list and say, "Okay, I could probably toss these words out when you really fine tune your instruction."

Rachel:
But I definitely, in reading some of the research around vocabulary instruction, I think it's really important to pre-teach vocabulary for whatever unit you're teaching. You're also coming into it, focusing on those words during your read-aloud as well, and then maybe hitting on them a little bit after the read-aloud, so definitely that before, during and after reading. But you really want to teach explicitly, "This is a word, say the word." So important for students to say the word back to you and activate that phonological processor.

Rachel:
Let's say the syllables in the word. This is what the word means. What does it mean? Just very explicit with what it means. Okay. Let's use it in a sentence. Here's an example. Turn and tell your partner, you know what this means. And also I'm reminded of something that Deb Glaser does, who created reading teachers' Top 10 tools. She creates this prior to the lesson, that would allow the students to fill in the blank. Let's say the vocabulary word is comb, like I'm combining my hair. The teacher would say, "My hair was really styley, so I needed to..." And then she'd wait for the kids to say, "Comb." And then she'd say, my hair or something like that. It probably wasn't a great example. Giving them that opportunity to hear that word in a sentence.

Rachel:
And I think another way that I really thought about vocabulary too, is kids are never too young to start that morphology word study as well. Even just thinking about suffixes or some prefixes that can really help kids pull up S word. So even if it's a word that they might not have heard before, like the word rewire, they might not know what is this wire for, a noun or a verb. But as soon as we put re- at the beginning, they're like, "Well, it's something again, so that must mean it's an action that's happening." And helping kids really focus on those word parts as well. So I think in part of assessing vocabulary, sometimes that's a really good piece to look at, do they understand what a prefix and a suffix are? So we can look at their understanding of decoding and pulling apart that word systematically. That's a huge part. Even in kindergarten and grade one. Right?

Hanna:
Yep. Agreed.

Rachel:
Yeah. That fix S code.

Hanna:
[Inaudible 00:23:33].

Rachel:
They see cats as a word.

Hanna:
Yup.

Rachel:
[inaudible 00:23:36] decodable. Why do we have an S on there? So many students just don't understand that it has a meaning to it. It changes the definition of the word. It changes the part of speech of the word, things like that. Lots of different examples.

Hanna:
Okay. The last little piece, how do we assess comprehension? Because this is another way. We've done that with the leveled readers. So what's the universal screener, when we think about Acadience, how does that look specifically at comprehension?

Rachel:
Yeah. Okay. So comprehension is assessed on Acadience or any universal screener typically with oral reading fluency. So that words correct for a minute is a strong indicator of comprehension, as well as fluency. Part of the oral reading fluency measure, so the student is reading a passage, a grade level passage to you, and then after they read it, they're supposed to tell you what they read about. And so, they do a retell. And so, that also helps you assess their comprehension and you give them a score. It's like you select 1, 2, 3 or 4. And it just rates how well they captured the events of what they read. So that's a Y piece.

Rachel:
And then, as far as let's say, you've had a student on oral reading fluency, maybe they're grade two student, they were accurate, they were fluent. So they were at benchmark. Their accuracy score was great. Their words correct for a minute score was great, but the retell was, they couldn't really remember or they just gave one sentence and couldn't expand on it.

Rachel:
So to me, right off the bat, I already see something I can help them with. I want to help them maybe hear a story, so work on that listening comprehension and then be able to tell me about it. So that's something I would want to model in class. "Hey, we just read this, let's summarize this together." And so, I'm modeling that comprehension strategy.

Rachel:
There are comprehension diagnostics. And so, that's the other piece that if you have a student who is below the benchmark there and their retell isn't great, and you're like, "I just don't know what this kid is missing. They listened to a story and they were able to tell me about it." Maybe they need work with syntax, and you look into those other strands of language comprehension. Maybe they need a little bit more morphology and vocabulary and that sort of thing, or connecting ideas across sentences.

Rachel:
And so, there are comprehension diagnostics out there. Acadience has one, it's called CFOL, C-F-O-L, comprehension, fluency and oral language. It's not free though. So it's definitely something that a school would have to buy. And then I also just recently heard about one called CUBED, C-U-B-E-D, which is free online. I think you just have to give your email to have access to it.

Hanna:
Okay. So two things that came to mind. We'll wrap this, but two things in Lyn Stone's book, Reading for Life. She has a whole chapter about comprehension and she really talks about not from the point of view of assessment, but when kids really do struggle with comprehension, coming back to what are the steps along the way to be able to understand? And it's really that decoding fluency, the automaticity, the accuracy, that's what's helping comprehension and Timothy [inaudible 00:27:13] in his book has a really good diagram of a bridge. And on one side, it's word recognition. The other side is that reading comprehension and in the middle, the most important connecting piece, is the fluency.

Hanna:
So I guess, by the time we get to doing those big in-depth comprehension, we're going to see it already way back in some of those other universal screening pieces, because we're going to see it in the fluency. We're going to see it show up in that word recognition very early on, in that maybe their alphabetical principle. We've got deficit areas and being able to just recognize quickly all of those graph themes and putting them together. But we're going to see that way before we get to, did they understand what they were reading?

Hanna:
So all of this completely makes sense to really look at those screeners. And I think the screener will save a lot of us time. When you think about some of those leveled readers and doing that assessment, that was taking, on average, like 15 to 20 minutes a book, and sometimes you have to do more than one book. So already, just looking at that universal screener, I think that's going to be a massive game changer for people to look at this skill specifically.

Rachel:
Yeah. And the universal screening measures are set up in a way that they mirror how reading develops. So when you brought up the bridge analogy, I think actually I saw this in Acadience at first, they had a little pond, and they had stepping stones and two parallel lines of stepping stones. And the one side was word recognition and the other side was language comprehension. And it was like, kids have to have one foot on language comprehension and one foot on word recognition to be able to get to that skill reading side of the pond.

Hanna:
That's awesome. That's a good-

Rachel:
Right?

Hanna:
I like that.

Rachel:
And when you asked me about comprehension, I'm thinking of this student is reading accurately and fluently. So we've ruled those. We've rolled out those word recognition skills, so we know it's comprehension. We know there's something missing in those language comprehension strands. And that's what I love about universal screeners is they allow you to rule out those skills that are missing versus saying, "I think it's probably phonemic awareness, because everyone struggles with that, so I'll give that assessment first." It really might not be that.

Hanna:
Yeah. And I think in Lyn Stone's book, she talks about really only, I think it was about 1% of students in this big study that they did only out of all of those readers, there was about 1% of students that really needed targeted comprehension instruction over and above what you may be incorporating daily in your time in your learning spaces. And so I thought that was pretty also telling, right?

Rachel:
It's telling.

Hanna:
Yeah, I think again, Dr. Deb Glaser shares in her Top 10 Tools that it's something like 80% of reading difficulties are decoding issues and those word recognition skills. So it does make sense to look to that click first.

Rachel:
Right. Right. Yeah. But the screener does give you... it gives you a path to walk down.

Hanna:
Yep.

Rachel:
We start here. Okay. As comprehension is low, we move down a little bit. Fluency is low, we move down a little bit. And so, we're moving down toward those more foundational skills, right?

Hanna:
Yeah. Okay. So for everyone listening, the Acadience, the screener is free online. The link is in the show notes, and also on the blog post when this episode airs. Thank you very much, Rachel, for spending some time with me today and just talking about making that shift, not just in our instruction, but also in our assessment, which will impact our instruction. So have a great rest of your day. Chat soon.

Rachel:
Thanks for having me.