Bad Measurements

As a policy guy, I’m a staunch believer in assessment and evaluation, but I hate bad measurements. Unfortunately, bad measurements are so much easier than good measurements, and recently I’ve been required to comply with two particularly bad measurements.

One has to do with ensuring credit hours for independent studies. The federal government is trying to crack down on diploma mills, so all colleges and universities receiving federal funding must ensure that an hour of academic credit corresponds to an appropriate amount of work. This includes credit for independent studies, which are trickier to ensure than a course that actually meets for the specified number of credit hours, and so in setting up an independent study for a student for next term, I had to “demonstrate” credit hour equivalency. I was puzzled as to how to do this, until I found out that all I actually had to do was to assert it, with a statement on the syllabus along the lines of “this course meets the credit hour standards of an in–class course.” Heh, and here I thought actual evidence was required.

And in my on-line course for a community college, they are evaluating on-line instructors in part by whether we’re logged into the course software system for enough time. A well-designed on-line course doesn’t need the instructor to be logged in for significant amounts of time. The lectures and ancillary materials are all prepared, uploaded, and need not be looked at again during the course of the term (unless an error is brought to the instructor’s attention); the course grading can be done online, but it can also be downloaded and graded offline; and communication with students happens almost wholly via email which mostly does not require logging into the course software (although the software does have an email function). A person could satisfactorily teach such a course while being logged in only a couple of hours for the whole term, and in logging in for longer periods of time does not enhance the value for students because it neither requires, nor is likely to promote, student interaction (students take on-line courses for convenience, so it is highly unlikely that you could hold regularly scheduled chat sessions at a time when many students would find it convenient to log in). The incentive here is clear: stay logged in while you’re doing other work, including other classes.

A year or so ago I was told I wasn’t logging in enough, and they wanted on-line instructors to have more contact with students, so I should ask their course software specialist for ideas. And so I did. And they gave me ideas on things that could potentially make a course better–videos, etc.–but nothing that actually involved being logged into the system for any more time, or actually interacting with the students. And when I specified that I was looking for ideas that would result in more faculty-student interaction, I never received a response. My educated guess is that they didn’t have any ideas, either.

In fact I do have an idea about how to increase faculty-student interaction in an on-line course; design the course poorly and have numerous errors in it. Students will flock to email to consult the instructor. As I have fixed the problems over successive terms of teaching this course, I get fewer and fewer emails because I have made the course better and better. But that’s not as easy to measure, nor is the actual educational quality of the course, so they’ll measure my time logged-in.

I’m logged in right now. I’m sure it’s doing the students a world of good.

Advertisements

About James Hanley

James Hanley is Associate Professor of Political Science at Adrian College and a Fellow of the Institute for Social Policy and Understanding. The views expressed here do not reflect the views of either organization.
This entry was posted in Uncategorized and tagged , , , . Bookmark the permalink.

8 Responses to Bad Measurements

  1. Murali says:

    I’m logged in right now. I’m sure it’s doing the students a world of good.

    Federal Inspectors: Dr Hanley, it seems that you admitted to logging on without having any actual contact with your students. How do we trust that you are doing your due diligence with respect to those students?

  2. ppnl says:

    Coursera classes have an online forum for students where they can interact with each other and the teachers. I recently did a python programming class there and it seemed to work well. I have started to learn python by self study several times but lost interest because python is so different from other languages I have used. Duck variables just seem like an abomination. I generally dislike things that hide the details.

    A weekly schedule of assignments and real people to interact with made it easy. We even graded each others programming assignments. I wrote this program and posted it in forum early in the course:

    http://www.codeskulptor.org/#user4-WSJ7lbXNdj-0.py

    Just click on the left pointing triangle at the top to run. Arrows to move and space bar to fire.

    Duck variables, hard formatting and the naked ass way they don’t close function definitions are still an abomination. Yet the utility is undeniable. I feel I’m being drawn into a life of sin.

  3. James Hanley says:

    I not only didn’t understand any of that, I’m pretty sure I lost the game, too.

  4. ppnl says:

    Yeah, unless you speak python you aren’t going to understand the source code. You were just looking at the CodeSkulptor interface. CodeSkulptor is just an implementation of the python language in I think Java. That way the program will simply run in your browser. A program you write is just a web page you create. Makes it easy to write and share programs.

    As for your inability to win the worlds easiest game of space invaders… either you have welcomed our new alien overlords or the lack of polish of the game has confused you. You win the game when there are no more w’s to shoot. No indication in game just no more w’s. You lose when an x bomb hits your gun. Again no indication other than the game stops responding. To restart you have to close the window and restart from CodeSkulptor.

  5. Matty says:

    I’m not even going to bother looking at the ppnl’s link because I know I wouldn’t understand it but would an online forum where there was evidence of students interacting with each other and the teacher be good enough, or do you have to be online at the same time?

  6. ppnl says:

    Dude, its just a space invaders game in ASCII. You don’t have to read the source code just play the game.

    I’m actually glad nobody here speaks python because it is pretty piss poor python code.

    Anyway I think you need a critical mass of students for it to work because you end up getting as much value from fellow students as the instructors.

    And trying to make everyone interact in real time would be horribly destructive.

  7. Matty says:

    Oh, OK I had a play. I lost

  8. Trumwill says:

    There’s a good quote somewhere something to the effect of “Tell me how a man is valued, and I will tell you what his values are.”… or maybe it was more straightforward “Tell me what you’re counting, and I’ll tell you what he’ll do.”

Comments are closed.