Category Archives: testing

More On Making Learning Relevant

Leave a reply

Sometime after I posted the last entry on the relevance of Algebra, I was paging through a book catalog and it seemed like each page had at least one book that focused on how and why literature mattered. Here are just a few of the titles:

I haven’t ordered any of them. I’ve already read Moby Dick and W.H. Auden and spent a lot of time teaching children Shakespeare. At some point I realized I was trying to turn them into English majors, when what I really wanted to do was help them learn to love reading the way I did. If, eventually, they found Melville and Auden and the Bard, so much the better. But to force it upon them meant it only led to the inevitable question of why they needed to read it in the first place.

It is an interesting side note that the authors of these books are writers who were probably English majors at some point in their lives so perhaps the lesson here is that, if you plan to become a writer, then reading literature is part of the career path.

And then there’s this blog entry from Edutopia just published today: Why Do We Need To Learn This? Allen Mendler offers strategies for answering the question that might diffuse the immediate situation but never gets to the heart of the answer which is that someone, somewhere decided that “this” was important for everyone to know and, as Mendler does point out, it is going to be on a high-stakes test:

Upon hearing the “When will I ever use this?” refrain, a high school teacher I work with tells her students, “I’m not sure because I don’t know what you want to be in your life. But if you give me a list of everything you plan to do and accomplish, I’ll do my best to let you know when we cover something that I think you might use.” When kids say, “I don’t know what I’m going to do,” her response is, “Exactly. You might need it next week, next year or never. But it is going to be on Friday’s test, not because I want to make you miserable, but because at the end of the year, it is going to be on the state test, and if you want to pass, you need to know it.”

So, you have to know it because I’m going to test it and later someone else is going to test it? I think this is probably the worst answer to the question but the most relevant in our high-stakes world and that just makes me sad.

 

Testing Zombies

1 Reply

As a defender of public education, I took Tom Woodward’s challenge Scott McLeod’s challenge seriously. I want to fight fire with fire in the war being waged against public education. Tom Scott outlines the seven steps of the strategy being used by organizations like the Chamber of Commerce. His blog entry is worth a read despite the depressing conclusion. Go, read…

i didn’t want to believe that public education could be beaten by such a blatantly cynical sound bite campaign. So, I set out to answer step for step using high-stakes testing as the target.

According to Tom Scott, step 1 is to get a snappy slogan. I came up with “Stop the Testing Tragedy.” Maybe not all that snappy but I don’t have the benefit of a focus group.

Step 2 is to create made up statistics to prove my point. Instead, I used real but sketchy statistics. Some 40% of students suffer from the anxiety that has been shown to influence test scores and cause other physical and emotional damage. The slide show outlines the sketchiness: most research happened prior to our current testing craze engendered by No Child Left Behind. And the studies don’t distinguish between classroom and high-stakes testing. Plus, the percentages are all over the board in the studies  cited: “Current estimates of the percentage of students in a classroom affected by test anxiety range from a low of about 1% to a high of over 40% (Cizek & Burg, 2006, p. 29).” So my stat is true but who knows “how” true. What I do know is that I’ve heard anecdotal evidence that kids are stressed out by testing from students, parents and teachers.

 

 

Step 3 is to come up with a graphic. I wanted to use the drooling kid from Ferris Beuhler but it was more gross than eye catching so I went for this one instead. She needs a speech bubble but I don’t have a writing crew so for now we just get the scream.

 

Step 4 involves maps…and there’s where I got distracted. I thought I would make a map of all the states that Tom Scott mentioned who beat Finland. Or states that had reasonable opt out policies. Or maybe all the states where there had been test protests this spring.   That last idea led me to the Rhode Island students who dressed up like zombies to protest state tests.

Testing zombies…now there’s an idea that could catch on. I went looking for copyright friendly images of zombies and what did I discover? The poster for Night of the Living Dead is in the public domain. That took the rest of the evening.

According to Tom Scott, there are three more steps: make report cards, use social media, spend boat loads of money. Educators are busy making real report cards, struggling to provide authentic assessment in a world that prefers letter grades and test scores. They certainly don’t have boat loads of money and what they do have goes to buying classroom essentials from crayons to tissues to books. Not to mention that they are spending their days with second graders instead of marketing consultants and research assistants. 

But social media is accessible to everyone. Unfortunately, school filters and fear mongers often keep educators from engaging with social media in professionally productive ways. Posting student work and classroom success stories could go a long way to telling a very different story about what is happening in public education when teachers are willing to shut the door and skip test prep for a few days.

But now may be the time to dig in and fight with whatever tools we do have. There was some buzz this spring as several educators very publicly left the profession because testing clashed with their principles. There is a ground swell of opposition to testing in the general public even those who do not have children in school. Recently, I was at a meeting of local historical leaders and it didn’t take long for them to lament how testing has led to a narrowing of the history curriculum that forced out any mention of the rich local history in a region that includes Nat Turner’s rebellion. I encouraged them to talk to their legislators and let them know that not everyone thinks testing is the way to figure out if our schools are successful.

Educators must speak up for themselves and those of us who believe in what they do, must also speak up. And, I’d like to believe we can do it not through incorrect, misleading information but by telling true stories of teachers and students and classrooms. And a few zombie posters…

The Danger of Data: A Charter School Update

Leave a reply

Interesting follow up to yesterday’s post about charter schools.  The Commonwealth Foundation points to evidence that more charter schools made Adequate Yearly Progress than traditional schools. It turns out that this positive trend was manufactured by the state’s Secretary of  Education by turning charter schools into districts so they have a lower standard to meet for AYP.

Diane Ravitch covered the story on her blog this morning, concluding, “This is the intersection of politics and education, where the data are adjusted for political ends.”

With a simple definition change, failing schools were now successful schools.  Their traditional counterparts did not have that same luxury so they continue to struggle to meet the increasingly stringent demands of the law. I’m reminded of a Pennsylvania school superintendent I interviewed several years ago.  He assured me that he knew his school would never reach the 100% pass rate but that he and his faculty was going to continue to work hard to help all their student succeed as long as they were allowed to do so. The deck is stacked against these educators in so many ways that the most amazing part is that they keep working every day.

Finding the People in the Picture

Leave a reply

This fall, I will be teaching an introductory qualitative research course. My own dissertation research used a qualitative methodology to learn more about how teachers plan for the use of technology. I interviewed and observed teachers at work in their classrooms with their students. I wrote short vignettes describing that work and the challenges they faced from high-stakes testing to inadequate access to resources. While I’m sure my research will not have much of any impact, I am proud of the way I represented the complexity of the classroom through the voice of the teachers.

For me, that’s the value of this kind of research. Certainly, quantitative research with its percentages and statistics and measures of error, is useful for wider “big picture” sort of research, providing access to general trends and suggestions for practices that might lead to greater success in whatever given area is being studied. But, qualitative research paints a different picture, of the people themselves, the ones who make saying anything definitive about education very difficult. I am often much more interested in those personal stories and insights than in the big picture ideas because they remind us that education is first, and foremost, about human beings.

If you’ve been following the news about the school in Rhode Island that had decided to fire all its teachers as part of its reform efforts, you’ve seen a glimpse of this tension between the big picture and the individual people. The latest news is that the administrators and teachers have negotiated an agreement and they will not be fired after all. My thoughts about the agreement itself are for another post, what I’m interested in here is the way the story plays out in the version I read at NPR.

You have to scroll all the way to the bottom to find the people in the story. The teachers are only present in the person of the union boss while the school district itself is represented by the Superintendents and a state administrator. They aren’t really “people” in my book but talking points who are saying all the right things about this agreement and the efforts they are making to improve education in their district. Even the Obama administration plays a role, but again, one that is preordained and peppered with words like “accountability” and “chronically underperforming.”

But there, in the last few sentences are the people: the parents and students who haven’t been involved in the agreement and yet who will be influenced by its outcomes.

The teachers largely have won the support of students and parents, many of whom believe the staff has been made a scapegoat for the woes of a high school in one of the state’s poorest cities. Norma Velez, whose 15-year-old son, Jose, is a sophomore, said she was pleased to see the teachers return. “When the teachers teach to students — some of them — they don’t want to cooperate with the teachers,” Velez said. “They just do what they want, and they hold up the rest of the students.” Julia Pickett, a 17-year-old senior, bristled at the description of the school as failing. “I don’t like that perception of us. I think we’re a great school,” she said. “Just one test score doesn’t determine whether a school is good or bad.”

Here’s that glimpse of the real people behind the “facts” of the story…the brief insight into the kinds of classrooms these teachers face each day. The momentarily glimmer of the idea that the human beings behind the numbers don’t see themselves as failures. And, in support of my own bias, the suggestion that teachers are not the only ones to blame but have been part of a wider failure of imagination throughout the education community that has developed simplistic, easy to evaluate definitions of student achievement and success. It does often get boiled down to a number–just one test score–and the human beings get lost.

A Pragmatist In a Progressive World

Leave a reply

This year, I have the opportunity to be part of an online professional learning community.  While I will be taking on the role of facilitator, I believe this will be as much a learning experience for me as well as for the other participants.  And, the opportunity has already gotten me thinking about where I fit into the sometimes confusing but always intriguing world of “educational technology.”

Here’s what I know:

Educational technology is about much more than just technology.  In a way, technology is the easy part.  It’s easy for me to show you how to use a flip camera to capture video or a digital microscope to find Abraham Lincoln on a penny.  It’s easy for me to post a link to a wonderful interactive website.  And while all these things may be cool, most teachers want more than just cool.  They want to know that the time and energy it is going to take them to set up microscopes or plug in projectors or to have them or their students create videos will have some positive influence on their students and their learning.  That’s the hard part: helping teachers figure out how to use these technologies in powerful ways in their classrooms.  So, while I may like to explore new technologies myself, my focus with others is on the educational part.  How/why/when to use those computers and gadgets and websites to improve teaching and learning.  This might seem like an elementary idea, but I still go to lots of “educational technology” presentations at conferences where the heavy emphasis is on the technology rather than the education.

Here’s what else I know:

I have a deeply held bias. I believe that technology offers ways to improve teaching and learning.  Even if it’s only because it engages the kids in ways that textbooks and lectures and worksheets do not.  And, most of the educators I talk to seem to share this two-part belief with me.  Part one: technology engages kids.  Part two: engaged kids are better learners.  But they also share a concern about doing it the right way.  They don’t want to just use technology for technology’s sake.  And, I find myself working with them in very practical ways.  Have you thought about using a smartboard to let your kids interact with a sentence?  Do you know that you can put a video in a powerpoint presentation to show to your kids?  Have you accessed the data from the student response system to better differentiate instruction? Have you considered having your students create a digital video or multimedia presentation as an alternative assessment?

I also use this practical approach when I work with technology coaches and school administrators in helping them to encourage technology use.  I’ve created a presentation called Strategies for the Non-Choir.  It draws from Rogers’ work in diffusions of innovations as well as Mishra and Koehler’s Technological Pedagogical Content Knowledge (TPACK) model to provide coaches with ideas for how to approach the early and late majority adopters who, according to Rogers, make up some 68% of the population.  I talk to the coaches about the need to consider the relative advantage of a technology as well as how compatible it is with what the strategies already used by a teacher.  In addition, as part of the workshop, we play the TPACK game where we match technologies, pedagogies, and content areas to come up with ideas for using technology in the classroom.

So, I am very much a pragmatist, trying to work with teachers where I find them, helping them use technologies in ways that support what they are doing in their classrooms.  This is a viewpoint that is often in direct opposition to the visionaries in the educational technology blogosphere.  They tend to be progressives who are looking past the current times to a different world where powerful technologies support student-centered, constructivist learning.  One of my favorites, Tim over at Assorted Stuff, summarizes the viewpoint quite nicely, I think:

The powerful tools we now have available make it possible to go way beyond simple reinforcing what we’re already doing. They provide communications links that enable teachers and students to connect with and learn from the world.

If all we do with the computers and networks put in our schools over the past decade is multiply the status quo, then we’ve wasted a lot of money, time and effort.

I know much of the crap I write is very idealistic, maybe even unrealistic. But while we are making small incremental changes, it would be nice to keep a vision of what education could and should be in the viewfinders.

I don’t disagree with Tim.  And I admire his idealism. I am also always inspired by Sheryl Nussbaum Beach. One of Sheryl’s most recent posts over at 21st Century Learning gives some great examples of how are kids are learning to learn on their own, and she calls to us to roll up our sleeves and get to work on creating a learning environment for them.  I try to keep her vision in my mind and for awhile I move into that progressive world.

But then I go to a school or talk to a teacher and hear about the sorts of barriers–time, access, not to mention high-stakes testing–that they face and how excited they get when someone gives them an interactive whiteboard or even just a projector and the pragmatist returns.   To borrow a phrase from Tyack and Cuban, we are “tinkering toward utopia.”  I think I’m more the tinkerer, standing with a wrench in my hand, rather than the utopian, envisioning the future.

Isn’t It Ironic, Part II

Leave a reply

Last week, I blogged about the report out of Chicago that shows that intensive test prep may have actually led to lower scores for students on the ACT test.  This morning, I opened US News & World Report and found their description of a a new study from Stanford University that what appears to work is a reward system, including pizza parties.  There was a 4 percentile increase in reading scores but none in math.

Margaret Raymond, the author of the report, says the gains are more significant when teachers and administrators work together to support the use of rewards. Successful schools included those that rewarded good grades and good behavior with such gifts as concert tickets and MP3 players.

I would be negligent if I did not report that the sample was made up of charter schools and this happened in what they call a “majority” of cases.  Could it be that rewards are only one part of the reforms taking place at these charter schools?  I have not read the report (pdf) yet so I can’t really comment on the study design.   You can read more about Margaret and the Center for Research on Education Outcomes and find additional links to reports about the report here.

Ironic, Isn’t It?

1 Reply

From the Consortium on Chicago School Research from the University of Chicago, a report that extensive test prep actually lowered student scores on the ACT. The report–From High School to the Future: ACT Preparation–Too Much, Too Late—found:

  • Low ACT scores reflect poor alignment of standards from K-8 to high school and from high school to college.
  • Test strategies and item practice are not effective mechanisms for improving students’ ACT scores.
  • ACT performance is directly related to students’ work in their courses.
  • Incorporating the ACT into high school accountability is not an effective strategy for high school reform by itself, without accompanying strategies to work on instructional practice.

They concluded that students are trying to cram in study for the ACT when the best preparation takes years of college-prep study and work.

I know that the researchers would probably not want to generalize to other types of tests, but I can’t help but wonder how this might apply to high-stakes testing in general.  I have a feeling sometimes that school has become just one big test prep program, with students and teachers spending each year cramming for the test, then beginning anew each September.

A Finnish Edutopia

1 Reply

I was planning to post about Finland after reading this article in eSchool News that described a recent visit to Scandanavia by US educators.  Finland was of particular interest since they are often first in international tests of math and science.   I never got around to the post yesterday because I spent the time I had learning more about Finland at Wikipedia and the CIA Factbook.  Then, this morning, Tim over at Assorted Stuff pointed me to a post about a series of articles related to Finland’s education system.  Combine all this with Sheryl’s 9 principles for implementing what she calls the Big Shift, and we begin to see how education can become more than just something students get through.

Tim writes,

While I’m sure there’s much more than meets the eye, their success seems to boil down to a high degree of trust for the students combined with high expectations for their learning.

I agree.  I also agree wholeheartedly with Ewen who points to the trust that they put in teachers.  Being a teacher in Finland is the equivalent of being a doctor or lawyer here in the states.  Only one in eight applicants gets into schools of education and teachers are widely respected.  They are given high levels of autonomy in their classrooms where the pedagogy is very much student-centered.  Hmm…is this the edutopia we’re always talking about?

I also agree with Tim’s comment that there is more than meets the eye, particularly in terms of making these international comparisons.  Finland is not the United States.  My research into Finland led me to see it as one big exclusive private school.  The incredibly homogeneous population equals that of Rhode Island and Connecticut.  Everyone speaks the same language, and, it appears, shares the same culture, values and history.  And, one of those shared values is education.

I don’t want to discourage us from looking to places like Finland for inspiration.  But, I also want us to recognize that America’s great experiment of educating everyone leads us to grapple with an incredibly heterogeneous population that often does not speak the same language or share the same values.

And, while at the national level, we may not seem to trust either students or teachers, at the local level, I’m seeing some of the “big shift” happening. I have been in several high schools this past year where teachers are part of the leadership and are implementing amazing changes in their classrooms, partly from a wider access to technology tools such as blog and wikis, and partly because they are changing the relationship they have with their students.  I would suggest that, rather than sending delegations to Scandanavia, we might be better served by sending delegations to those schools to highlight what we’re doing right in this sometime suffocating standards-based world in which we live.

Just Makes Me Sad

Leave a reply

In the August 15, 2007, edition of Education Week, Amy H. Greene and Glennon Doyle Melton describe their efforts to meet Adequate Yearly Progress at Annadale Terrace Elementary School, a Title I school in Fairfax County.  Their ideas about incorporating the genre of test language are laudable and, evidently, successful.  But their reasons for doing so make me a little sad.  When they asked themselves why their kids needed to pass the state tests, they concluded, “After much soul-searching, we had to admit that state and federal pressures were not the only reasons students needed to learn to pass tests.  test-taking is a life skill.” (Since I’m prepping for comps right now and had to take the GRE to get into grad school, I agree with that.)  But then they write: “While we believed that the test was biased against our students, many of whom came from low-income families and spoke English as a second language, we also knew that much of their academic and professional futures would be determined by their performance on similarly flawed tests.”  That comment came off the page at me like a punch in the nose.  While I applaud their efforts to help their students achieve, I hope that they are busy fighting against the trend in using one flawed test to judge that achievement.  And, I hope that the other tests they mention–such as those for certified public accountants and teacher licensure–are not all flawed.  Have we just gotten to the point where, even though we know tests are not the best indicator of success, they are just part of the nature scheme of things?

NAEP and Reading Research

Leave a reply

I just finished reading Gerald Bracey‘s Reading Educational Research: How to Avoid Getting Statistically Snookered.  It was an excellent review of my quantitative research class as well as a very real-life application of the information from that class.   Bracey spends several pages discussing the National Assessment of Educational Progress, or NAEP.  This test, which began in the late 1960s, was designed to be a national descriptive assessment.   However, Bracey points out that in 1988, Congress changed the law under which NAEP is administered to allow state comparisons and established a governing board, whose first president, added achievement levels to the test.  The test became a prescriptive assessment.  According to Bracey, “the achievement levels–basic, proficient, and advanced–were set ludicrously high and led to demeaning statements about the competence of students and, by implication, the competence of teachers and the quality of schools” (p. 149).  Many groups have suggested that the achievement levels are “fundamentally” flawed.  The problem, Bracey says, is with the word “proficient.”  This word differs from state to state.  But some folks, including Diane Ravitch (whose new book is on my shelf), would like to see a national definition and believe NAEP would be the perfect vehicle for getting at that definition, something Bracey says would be a disaster.

After I closed up Bracey, I reached for this week’s Education Week.  There on the front was a note about the recent results of the NAEP Economics test.  The headline indicated that the results were better than expected and the article inside indicated that the results showed that students had a pretty good understanding of the basics of economics.  They only ran into trouble with the details such as concepts like interest rates, gross domestic product, or the role of the Federal Reserve.  After reading Bracey, I had to wonder how many of the students taking the test had had direct instruction in these concepts.  According to the Education Week article, “87 percent of test-takers said they had received some kind of exposure to economics content during high school.  Sixteen percent reported they had taken advanced economics; the greatest proportion, 49 percent, said they had taken general economics, 11 percent said they had taken a business or personal finance class; 12 percent said they had been enrolled in some kind of combined economics course; and 13 percent said they had not taken any economics course.”   The article goes on to quote a federal transcript study that showed that 66 percent of high school graduates had taken an economics class in 2005.  So, this really is an example of an invalid test, since it is testing material to which the students may or may not have been exposed.

The article quotes Darvin M. Winick, the chairman of the NAEP Governing Board, who helps perpetuate the idea that our schools are failing.  He says, “While there is clear room for improvement, the results are not discouraging.  Given the number of students who finish high school with a limited vocabulary, not reading well, and weak in math, the results may be as good as or better than we should expect.”  OUCH!  There’s finally some good news about education, and this guy manages to turn it into an insult.

The issue of Education Week also has a general article about using NAEP as a way of comparing the states.  The fundamental problem for now seems to be that there needs to be more information about “exclusion rates,” that is the number of students that each state keeps from taking the test or to whom they provide special help or accommodations.  This information, according to the article, will be more visible in future NAEP reports.  And, the governing board will press for nation-wide inclusion policies for the exams scheduled in 2009.

One other criticism that Bracey has of NAEP that I think is particularly important is how meaningless the test is to students.  Since they are not held accountable for the test results since they can’t be traced to an individual student,  getting kids to take the test seriously is a problem.  Bracey quotes Archie Lapointe,  a former executive director of NAEP: “Yes, the problem with NAEP is keeping students awake during the test” (p. 147).

The National Center for Educational Statistics has a NAEP website that has lots of information about the history of the test.   And, here’s a Bracey post from February 2007 about NAEP testing.  I added his blog to my aggregator!  I LOVE it when people like him use weblogs to comment on contemporary education issues.