Back when I taught CS, I had a colleague who did this. He ran the program and compared the output char by char to what he expected. The assignments went into great detail on the expected format of the output (column-by-column). I believe this was all he did, no check of the source code. The sad thing was he was the chair of the department when I first got there. Fortunately, he retired soon afterward.
You know, this isn't a great way to teach in general, but I wouldn't mind it being "in the mix". There's a fair number of people out there who can program but don't have the discipline to understand a somewhat exacting specification.
Someone else writes of all assignments getting a public test suite, which sounds great most of the time but is even worse in making sure that students can understand what an assignment is actually asking.
I'm about to start teaching a middle school competition robotics elective. One of the two difficult grading measures is going to be tests (both closed book and open book) about what the rulebook says and means. (The other will be documentation. All the rest will be subjective and easy and squishy and "easy A").
