Thursday, September 26, 2013

Assessment and iPads: Questions, Questions, Questions!

How do we know what students know?

That is the million dollar question isn't it.  How much money is spent trying to figure out what our students have learned?  How much political wrestling between states and districts go into trying to see what is in the heads of the students?  How do we measure knowledge?  What do we measure?  Should we use multiple choice tests?  She we make students write essays?  Should we make students perform?  Do we try to find out how much a student has memorized?  Or do we want students to utilize knowledge and demonstrate their reasoning?  Those are the big questions that drive educational discussion.

I was told by my own high school chemistry teacher, Mr. Rice,  when I went back to teach at my own high school in Downey California, "The pendulum always swings from one extreme to the other.  When you get hit by the pendulum three times, it's time to retire!"

Being born in 1965 makes me not quite a baby boomer and not quite a generation x'er.  I am often stuck in the middle of extremes, so I will go with balance between different approaches to things.  I like to use multiple measures in multiple situations.  Here at Cerritos a few years ago we used the term "visible knowledge".  I like that term, knowledge I can see.  We began a conversation  ten years or so ago about learning outcomes which at the time it seemed related to visible knowledge.  Our learning outcomes for students were things that they would create, that would demonstrate their knowledge.  It was a product we could look at that maybe collected different types of work that we could look at from different angles.  This work would show knowledge but also present it in a larger, more whole way.  When I taught high school we used portfolios as a way to evaluate student growth and knowledge.  Portfolios were collections of "best work" that were diverse enough to give us a big picture view of the student.  It showed how well the student could integrate knowledge from different areas and present it in different ways.  A good analogy would be an artist's portfolio.  A good portfolio would demonstrate that the artist has multiple skills and maybe even could tie different works together with a common theme.  Portfolios also gave students a sense of accomplishment.  It was not just some score on a multiple choice exam.  It was tangible.  It was visible knowledge.

It seems as though the outcomes discussion has taken us away from integrated tangible demonstration of knowledge and brought us to discrete learning objectives that are just like those I used when I first started teaching high school 25 years ago. Duck, the pendulum is coming!

But maybe with innovative technology we have began to find our way back to balance.

iPads will not solve every problem.  Ha! Indeed, hey will create new ones.  Nevertheless the technology will allow us to jump ahead, and by trying new means of assessment and evaluation we might be able to bypass some of the arguments that have been blocking our progress.  Innovation often does that.  I would like to take a minute to define some terms.

Authentic

I like structures assessment that have two characteristics.  1) The form of assessment should not be that different from the actual learning experience.  For example, we don't normally teach in a multiple choice setting.  In many of my educational experiences I was "taught" by a lecture and then I was evaluated by a multiple choice exam.  Many times the questions were not even related to what was covered by the lecture.  Here is an example an authentic form of evaluation.  Students in a lab may perform several lab techniques to solve different practical problems.  In an evaluation or exam or test a student would be given a similar assignment that may involve needing to show mastery of one or more of the techniques learned over the semester and one or more of the algorithmic problems.  The test may even integrate several techniques and require utilization of several types of the reasoning skills learned and use these to solve a new problem.  2) It should be difficult to tell the difference between "normal" everyday learning and when the process of evaluation is happening. In real authentic evaluation, the method of measuring knowledge is so close to the method of instruction that an outsider cannot even tell the difference?  Even more the students' knowledge may increase during the authentic evaluation.  Learning never stops.  I don't think multiple choice testing, although there is a place for this, is very authentic.  I think it is quite different from the teaching and learning experience and I don't remember learning much during a traditional exam.  Imagine learning just as much during a test as you did at any other time during the class.  Please bear with me for a couple more definitions.

Evaluation vs. Assessment

So far I have not distinguished between assessment and evaluation.  So let me do that now.  I will just give you the way I use these terms and hopefully you will thoughtfully consider how you use them.  I think that although evaluation and assessment overlap, they are different.  For me evaluation is a test or exam or assignment that gets graded.  I am evaluating the student's progress or knowledge at a particular point or over a particular period of time.  Assessment to me goes on constantly.  Assessment tells me how well I, the teacher, am doing.  Do they seem to "get it".  Is my method of teaching working? Do the students understand?  So assessment could be looking at the level of performance on an exam and saying, "Too many performed poorly on that question, I better reteach it or find some  other way to help them do what I want them to do."  Perhaps I am assessing an assignment and how well they learned what I wanted them to learn or maybe I am assessing a book to see if they learned from what they read.  Often I ask questions in the middle of a lecture or lab to see if the students are understanding.  I read their body language, I walk around the room to see what they have on their paper, or iPad.  If I am not satisfied I make the change.  To me that is assessment.  Evaluation is for the grade.  Assessment is a gauge to see how my current students are learning in the current environment and situation. Assessment and evaluation overlap.  But that is how I distinguish between the two.  Assessment is not a euphemism for the dreaded examination,  a euphemism we will just have to replace in five years.

Formative vs. Summative

What I am describing as the day to day minute by minute checking on my students is what I think many people call formative assessment.  I am assessing how my students are "forming their knowledge" and figuring out what I can do to improve that learning process.  For me, formative assessment demonstrates that teaching is all about the relationship between the student and the teacher.  It is a both/and process.  And it is balance.  As a teacher I must know my students and care about them.  I am so profoundly grateful that at the place where I teach the previous sentence describes what drives the vast majority of my colleagues!  Teaching and learning is a relational experience.  So let me say loudly technology is not a substitute for the relationship between the student and the teacher!  For me summative evaluation is that test or assignment that gives me the big picture view of a student's knowledge and output over a broader period of time.  The summative evaluation could be an exam at the end of a few weeks or it could be a project we have been working on for some time.

Good people disagree on all of this.  I am just sharing my own experiences.

Finally, to the iPads!

Today was another big day for me because I gave my first "digital quiz" in my Organic Chemistry lab.  At the end of each lab experiment every professor in our chemistry department gives a lab quiz on the previous week's experiment.  Today I "cracked the mold" a bit.  I have stated in this journal before that I think it is good if students not just write their responses to questions but also get to explain audibly what they have learned.  I think it uses a different part of the mind and requires different kinds of thinking and different skills.  It seems to me that the students think that when they talk with their voice, they better know what they are talking about.  So in today's quiz I asked my students some of the same questions I might ask them on a common paper quiz but this time they had to create a screencast using their voice, photos, drawings and text.  They were told to use the Explain Everything app to make the presentation and them upload it to Youtube.  I knew this might be stressful for some.  So, for the first quiz like this I let them choose if they wanted to work alone or in pairs.  I also let them use their lab notebook but not the lab textbook.  I gave them 60 minutes to make a 5-7 minute presentation.  Here is a copy of the quiz: (If you click on the image of the quiz you can get a larger readable version.)


As you can see I included a rubric.  I will talk about rubrics in a minute. But when I handed out the quiz I tried to use a little humor to keep the students from getting too stressed out.  I want them to keep thinking that they have a good chance to succeed.  If that is the case then the stress level will be lower.  I also think trust is important here.  When doing something new like this I think the students have to trust that I am not going to destroy their grade.

As soon as the students got the quiz and understood the expectation they really got serious and started having an intense discussion.  I was quite surprised by this.  They took this very seriously and their discussions went to a very deep level.  That does not frequently happen in the lab setting.  I could tell they wanted to get it right.  And since they both had to talk on the video, they both had to understand and agree on what they were saying.  I think the team approach went much further than I had expected.  They also planned out their presentation.  They made a rough outline and tried to be organized. I was very impressed by this effort.  They really dug in and invested 100%. Not only did this exercise allow me to have an end product that I could assess and evaluate but being able to watch the process the students went through was a huge added bonus! Visible knowledge.

Rubrics

So what is good work?  What is an A or a C?  For me that might be the hardest question I as a teacher have to answer.  Most teachers I know take that very seriously.  But it kind of boils down to another question:  "What do I expect of my students?" Do I really know what I expect of my students?  Do my students know what I expect of them?  Those last two questions really get at the heart of the matter.  I want to have realistic expectations and I want my students to know very clearly what those expectations are.  Now, I have to make a huge confession.  This is a hard one.  I have to say that when I try something new, I don't always know what I really expect and I don't always know how to tell my students very clearly what my expectations are.  Usually the first time I have tried some innovation in my teaching I did not do a very good job of telling my students what I expect.  I must say that over the last 25 years I have had the priveledge of working with some of the finest young students!  So being purposely vague with my expectations has often worked.  I have gotten some very good work.  Some of that work I have displayed in this blog.  But once I have seen what top students can do I think it is my responsibility to communicate what good work is to the students that might not have performed as well.  So I usually make a rubric from the best student work the previous time I assigned it.  Sometimes I am purposely vague so the students have to reach high and use their creativity.  Today with so many new forms of technology and new types of skill that my students have I want to see what they come up with.  Maybe they will think way outside of my box.  They might come up with something way better than I ever imagined. Most of the time I am real clear on what I want so that every student has a clear understanding of my expectations.  Often I will show students an example of quality work.  I especially do this when it comes to writing lab reports.  I think one of the keys to life is having realistic expectations and communicating those expectations very clearly. So for a quiz like the one I gave today I tried to very clearly express my expectations.  Here are two examples of the students' work. There are some errors.  But remember they only had 60 minutes.  I am quite impressed with what they accomplished.  (Ok some of them took a little longer than an hour.)  But for the first time at this I thought they did a good job.








When it comes to learning and measuring learning iPads are just tools.  But sometimes tools can help us get much farther with them than we otherwise could have without them.  Technology is not the answer to all my questions.  But I think I see evidence of student learning and not only is this a good tool to evaluate their knowledge the very process itself of creating these screencasts was a wonderful thing to see.  I could witness the students learning as they were "tested".  So for me this was a form of authentic assessment even as it overlapped with evaluation.

No comments:

Post a Comment