Do the students finish the tests or does the test finish the students

I tweeted this the other day:

What led to the tweet was a discussion I was having with some students about not having enough time on tests which led to a discussion of having to drop everything to spend every waking hour on a project.

Let's talk about tests.

I'm not saying that tests are the best forms of assessment - most of the time I'd much rather have students work on projects. There are, however, times when tests make sense or are otherwise appropriate.

In any event, writing a test is hard. Rather, writing good test is hard. It's certianly easy enough to put a bunch of multiple choice or short answer questions on paper and it's easy enough to give a hard equation to solve or some code to write but creating a good test is a task and half. You first have to figure out what you're trying to assess - memory, thought process, synthesizing concepts? Then you want to construct questions that give you insights into your students knowledge and thought process.

Some things that I consider when putting together a test:

  • Does it ramp up in difficulty - that is, are there some "gimme questions" and some challenges.
  • Are questions all or nothing - if a kid doesn't see things my way are they dead in the water.
  • Will the test repeatedly penalize or reward the same concept over and over again on the test.
  • Do I cover all the concepts I want to assess.
  • Do you make kids waste time with boilerplate code.
  • Do the questions take so long to read and digest that there's little time to form and write down answers.
  • Do the answers convey anything about the students thought process or just correctness.
  • Is it easy or impossible to grade and grade fairly.

What about length? To me a well crafted test should be completed by the average student with a few minutes to spare - enough time to check a couple of answers. This is not to say that they'll ace the exam, just finish it. Far too many teachers make tests an assessment of speed and accuracy at speed rather than understanding. That might actually be important in certain contexts - preparing for the APCS-A multiple choice section as an example but in general, it's not a good way of assessing what a student really knows.

It's also important that the the average student can achieve a score that you expect from an average student. That's probably in the 80s on a 0-100 scale or a B. Yes, I know, C is supposed to be average but with grade inflation being what it is…

You should NOT give an assessment where the average score is something like 17 out of 100 with the top student earning a 37. Sure, you can curve it but it also places a lot of stress on the students. You might do this from time to time - you might misjudge the difficulty of a test or your class but it shouldn't be a regular occurence. Teachers sometimes forget about the psychological affect that a unfair test can have on a student even if the teacher "fixes" it after the fact.

Don't be afraid to experiment or have some fun.

It's also ok to try different things. One year, having just completed a unit on Cellualar Automata I decided to give a quiz. I figured it would take the kids 10 to 15 minutes so I gave them 30. The quiz was something like the following:

You have 30 minutes to compose something on a sheet of paper that when I review it convinces me that you know something about the Cellular Automata Unit we just completed.

Some kids loved the quiz, some hated it. The ones that hated it had been trained for years as expert standardized test takers and this level of freedom really freaked the out.

Another time, I gave a multi page test of serious questions mixed with crazy shenanigans. Question one would be some CS problem followed by some instructions like "stand up, do 5 jumping jacks and sit down" or "shout out your favorite olde timey exclamation" or even "stand up, if or when you see another student standing, switch seats with the and continue the test." The test started with explicit instructions not to read ahead but to read and do each question in order. The last page was the answer key and I asked the kids to self-grade. Interestingly enough the grading was pretty honest. After that one, I received a few apology emails from kids who read to the end first encouraging me to give them failing grades for cheating. Wow, I wasn't expecting that. The test was something of an end of year goof. The CS questions were really easy - I wanted to reward them with something silly and easy - a guaranteed A after a year of hard work.

Tests to drive instruction and future practice

As a final point to ponder, tests shouldn't only be about grades. A well crafted test should drive instruction. Kids will get answers wrong - will your questions be crafted so that you can gain insights into why the got them wrong.

In an early class you might notice things like:

  • kids printing rather than returning answers
  • kids not understanding scope
  • kids having difficulty with idioms like i=i+1
  • kids needing more scaffolding to approach a problem

This can drive instruction moving forward.

Over time you'll also learn how to fine tune your tests and other assessments.

Next time, we'll talk about projects

Unless of course I get distracted by another blog topic or shiny object.

Using Emacs 48 Silversearcher

A couple of days ago I wrote about on my lab grading workflow. In the post I mentioned that I used Emacs to easily navigate between student folders and files so I can actually look at their work in addition to their programs output and test results.

The key is a combination of dired and ag, Emacs's interface to the Silver Searcher which is something like a recursive code grep on steroids. I also briefly try to show wgrep in the video which allows you to edit Emacs grep buffers, effectively allowing you to make changes across a set of files all at once. It ended up not working with silversearcher - if anyone out there knows why, please chime in.


No Magic Bullets - Discovery Learning

The most recent NYC CSTA chapter meeting was "How do I assess CS?" I wanted to go but it's been such a crazy month I was just too wiped out. Fortunately, the meetup was recorded. I was able to watch the first half this morning while working out on my stationary trainer which leads me to today's rare Saturday morning blog post. The meetup consisted of two panels. The first was made up of K12 teachers who taught CS. I can't talk about the second panel yet since I'm only halfway through but there were a couple of points made by my friend Katie Jergens who teaches at Dalton that I thought were worth talking about.

The first was when Katie noted that through working with the panel, one of her takeaways was that:

Giving students an explicit strategy for solving a problem - "this is how I would do it….," - giving them an explicit strategy first and then giving them a problem for which that strategy would apply is much more effective than having them discover it on their own.

This was refreshing to hear after being beaten over the head on discovery learning and constructivism for the past five or so years as being the magic bullet for teachers. I shared more thoughts on this a few weeks ago when I wrote about a SIGCSE paper presented by Bootstrap World that I very much liked.

Katie went on to talk about how the group found that while "discovery learning" can be effective it can also be frustrating and lose kids along the way. It also takes a lot of time and preparation, something that's short on supply in most public schools.

This made me think about the recent flurry of discussion around the debunking of learning styles which had previously been the magic bullet. When I started, the secret sauce was cooperative learning.

Of course any good teacher knows there's no magic bullet. You fill your tool belt and chest with as much as you can and use what's best based on your strengths as a teacher combined with what will work best with your students.

Another point that Katie made that I found refreshing had to do with something she does with her classes. A good portion of a student's grade is based on what she calls a "booster." Each student has to work one on one with her on some project. The student either scores a 0 or a 100 - the student keeps working with the direct support of the teacher until it's perfect.

The important thing that Katie said with respect to this is "I can do this because my largest class is sixteen students." No way could this ever scale to a public school where a teacher can meet with over 150 students a day and barely has time eat lunch let alone work one on one with a student in a suitably quiet place like an office. The refreshing thing to hear was the acknowledgment of the fact that what works in a rich private school won't work in a public school.

All to often education and yes, CS Ed is driven by people who really have no clue. They're anointed as thought leaders but they don't walk the walk. Some have some knowledge and experience but many don't. Until you've gotten a few years under your belt in a strap cashed public school that takes all comers, don't tell them how they should do it. Katie didn't - she acknowledged the problem - too bad so many others don't.

I'm looking forward to watching the rest of the video during a future workout and might have more to share then. For now, take a look for yourself.

Unit Tests Grading Workflow

I've talked before about unit testing (here, and here). My premise is that in addition to being an important industry technique, it's a sound practice for students studying CS. I also contend that it can make grading easier on the teacher. Maybe not as easy as an auto-grader but those have their own problems.

Since I spent most of today grading I thought I'd share my current workflow and how unit tests have made me more efficient.

I have each student make a single GitHub repo for all individual homeworks, labs and small projects. They'll make additional repos for larger and group projects. They fill out a Google form to give me the repo link. I download the form and end up with a CSV file something like this:

Doe, John,
Doe, Jane,

My goal is to clone each students repo into a directory and then throughout the term, they add assignments to their own repos using the names and specifications I give them. For example, after assigning three assignments, I could pull everything from their repos and I would have a directory structure something like this:

├── doe_jane
│   ├── hw_01
│   ├── hw_02
│   └── hw_03
└── doe_john
    ├── hw_01
    └── hw_02

To get there, I have to change the csv file to a small shell script:

git clone doe_john
git clone doe_jane

I usually do this in Emacs with a macro.

Now I'm ready to go.

Whenever there's a new assignment, I just have to update all the repos. I go into the root hw directory and loop through all the subdirectoryes:

cs /path/to/hw_root
for i in `ls`
  cd /path/to/hw_root/$i
  git pull

At this point I could go into each repo but we can do better. Since I have all of my students submit a Makefile with a default target to build an executable named main and also a target named tests which will build an executable to run the unit tests named tests I do this instead of a straight pull:

cs /path/to/hw_root
for i in `ls`
  cd /path/to/hw_root/$i
  git pull
  cd hw_03 # or whatever assignment I'm grading
  make tests

Now, if I want, I can go into each directory, run the tests by typing ./tests and the hw or project with ./main. I can also add a couple of lines to the for loop above like:

echo "$i Test Report\n----------------\n" >> /path/to/hw_root/test_report
./tests >> /path/to/hw_root/test_rport
echo "----------------------\n\n"

This gives me a single file with everyone's test results. Either by doing this or by running the tests in each folder manually I can quickly see what works and what doesn't.

While I'm doing this, I have Emacs up and with dired mode and more specifically ag-dired-regexp which lets me easily navigate to any student's files. Combined with the test results I can quickly evaluate the entire assignment.

Put all of this together and it makes grading somewhat bearable. I can work through well written, correct assignments extremely quickly and drill down into the other projects efficiently as well.

Enter your email address:

Delivered by FeedBurner

Google Analytics Alternative