This week I've been reviewing the solutions of assignments submitted by student of our program transformation course. One of the things that strikes me again and again is how hard it is for students to get a grasp of the problem that they have to solve in an assignment.
In the last few installments of our program transformation course, the students had to develop a program instrumentation that traces the number of calls for every callee/caller pair for one of the assignments (we usually have about 10 assignments). Last year, we just described the problem in the assignment. The solutions of the students were ok, but not really exciting. They forgot to handle all kinds of cases, and some solutions even didn't terminate for some input programs.
This year, I included a set of tests in the assignment, which illustrate most (but not all) of the problems in this program transformation. Surprise, surprise: the students suddenly were able to handle all the issues illustrated by the testsuite that I provided. However, obviously I did not give the students all tests (evil grin). Indeed, several solutions could not handle the tests that I did not provide.
Most students don't write test. Worse, if they do test, then they create a single file and modify the test to check a new situation that they might have discovered. In this way they don't build up a nice testsuite. The important part of testing is that they can be repeated automatically, not that you run a test once! I'm trying very hard to convince students to test their code properly, but they don't seem to understand the need for it, so you typically get questions like "Is 10 tests enough?". I'm afraid that this is not a problem specific to students.
This might not be very surprising to you, but I am surprised how clear the results of this small 'experiment' are (I did not do this experiment on purpose). I wonder what the implications of this should be for education. Clearly, having a grasp of a problem is the most important part of the solution.
4 comments:
You'll never stop preaching about the importance of unit tests, will you?
I must agree with you of course, but you also have to admit that your hundreds of unit tests keep on turning you on when it says something like:
---------------------------------------
results testsuite Java Expressions
successes : 69
failures : 0
---------------------------------------
;-)
Oh yes, it does satisfy me, that's why it so much fun to write tests ;) . However, this is not strictly related to the number 0 in the report you show. I get particularly aroused if a test fails that used to succeed ;) . It shows the point of having an automated testsuite.
Well I'am a student, so I understand the students a bit. For most assignments, you'll get a specified instruction, precisely stating what demands the excersise has.
If you follow these demands, then you will get a nice grade. Good enough to pass. Students are used to working like this.
This caught my eye: .. some solutions even didn't terminate for some input programs.
In the Metaborg paper, if I remember correctly, you mention Stratego doesn't make any guarantees on termination. I imagine a stictly terminating rewriting engine would be less expressive and you feel it isn't that important in practice (am I right?).
However, even if they were bad meta-programmers, some students managed to produce non-terminating solutions for some inputs (emphasize mine, this is the part that makes it especially tricky of course). Can you say something about that?
I know in theoretical research, termination of various rewriting systems is a very important topic.
About the students: I can second glimi, both as a student and as a student assistent. The gross of them isn't interested in anything that's not covered by the exam.
On one hand, I can understand this (I feel this way too for some courses). On the other hand, some topic are so interesting, how could you not want to know everything about them? ;-)
As for teaching some topic, it's very disturbing when students have the attitude of just wanting the points. But it's also a challenge to enlighten them ;-)
Post a Comment