[LON-CAPA-admin] Scantron grading problems

[Todd Ruskell]

All,

Just a followup.  Stuart's suggestions helped a lot.  It turns out there
was an extremely ill-coded problem included in the exam.  The author
author of the problem has now been suitably educated in proper
programming methods, so hopefully we can avoid this particular nasty in
the future.

Todd

Todd Ruskell wrote:
> Stuart,
> 
> Thanks for the pointers.  The instructor has been playing a bit more
> with this, doing just what you suggested about using only a student or
> two as a test.  We think we've narrowed it down to a single problem that
> is causing our difficulties.  For now, it seems that if we delete that
> problem, and the associated scantron entries, the grading works fine.
> We can then grade that one problem "by hand."  Once that's done we'll
> try to figure out what went wrong that problem.
> 
> Todd
> 
> Stuart Raeburn wrote:
>> Todd,
>>
>> When the raw data file was re-uploaded, it will have overwritten any
>> existing file (if indeed one was written to disk during the initial
>> (failed) upload attempt, if you used the same file name.  The scantron
>> grading screen provides an interface to download scantron data files
>> uploaded to a course - the "Download: Show List of Associated Files"
>> button, so this could be used to compare the raw scantron data file with
>> the original data file provided by your scoring office.
>>
>> LON-CAPA does not currently provide a mechanism to "wipe out" all
>> grading information for a particular problem instance in a course. 
>> However, regrading a sequence using scantron grading will cause new
>> submissions to be recorded for each student, and the last submission
>> will be the one which determines the correctness, and the points awarded
>> to each student for each problem part.  A side effect of regrading via
>> scantron is to increment the number of tries (i.e., the number of tries
>> for each problem part would become two, which might seem odd for a
>> scantron exam).
>>
>> My recommendation would be to download the existing raw data scantron
>> file, copy it, and then remove all except the record for one student.
>> Upload this "reduced" scantron data file to the course with different
>> name, and then regrade with this file (this will cause new submissions
>> for just that student).  You'll then be able to compare the submission
>> history for that one student, to see whether the regrading made a
>> difference, particularly for problem 14 onwards.  If regrading appears
>> to resolve the issue, you can then regrade for the remaining students.
>> If the issue is not resolved, if you can provide me with temporary CC
>> access to the course, and send me the contents of
>> /home/httpd/lonTabs/scantronformat.tab on the server where scantron
>> grading has occurred, I can work with you to resolve the problem.
>>
>> I have recently worked on a utility to allow a course coordinator to
>> compare the raw scantron data record for each student, with the
>> corresponding submission record stored for that student by the scantron
>> grading process (it compares the last submission data).  This utility
>> will report any discrepancies found (and which students are affected),
>> and is expected to appear in some form in 2.7.  Contact me off list if
>> you'd like to have a pre-release version of the utility run for this
>> particular exam.
>>
>> Stuart Raeburn
>> MSU LON-CAPA group
>>
>> Quoting Todd Ruskell <truskell at mines.edu>:
>>
>>> Hi,
>>>
>>> We're having some problems running some Scantron exams with version
>>> 2.6.2.  At some point while uploading the scantron file the first time,
>>> LON-CAPA spit out a long error report (I can send it on individually if
>>> people are interested).  Apparently it did this without
>>> notification to the instructor.  Additionally, at some time during the
>>> initial upload, the client machine got rebooted.
>>>
>>> A second attempt to upload scores resulted in no apparent errors.
>>> However, when looking at the exam statistics, after about question #15,
>>> the number of responses quickly diminishes, and LON-CAPA reports no
>>> students out of 493 got the last question (#22) correct.  In looking at
>>> the raw scantron file, the statistics on these last problems are not
>>> correct.
>>>
>>> So the question:  It appears that something probably has gotten
>>> corrupted, but what?  Is it possible to relatively easily wipe out all
>>> grading information for this exam and start over?  In the exam all
>>> problems have 5 responses except #14 and #15, which have only 4
>>> responses.  But we've done that kind of thing before, with no problems.
>>>
>>> I should also point out that an exam in a different course was graded
>>> with no apparent problems the same day.
>>>
>>> Help!
>>>
>>> Thanks in advance,
>>>
>>> Todd
>>>
>>>
>>> -- 
>>> Dr. Todd Ruskell
>>> Senior Lecturer, Department of Physics       Office:  Meyer Hall 326
>>> Colorado School of Mines                     Phone: 303-384-2080
>>> 1523 Illinois Street                         Fax: 303-273-3919
>>> Golden, CO 80401
>>> _______________________________________________
>>> LON-CAPA-admin mailing list
>>> LON-CAPA-admin at mail.lon-capa.org
>>> http://mail.lon-capa.org/mailman/listinfo/lon-capa-admin
>>>
>>
>> _______________________________________________
>> LON-CAPA-admin mailing list
>> LON-CAPA-admin at mail.lon-capa.org
>> http://mail.lon-capa.org/mailman/listinfo/lon-capa-admin
> 

-- 
Dr. Todd Ruskell
Senior Lecturer, Department of Physics       Office:  Meyer Hall 326
Colorado School of Mines                     Phone: 303-384-2080
1523 Illinois Street                         Fax: 303-273-3919
Golden, CO 80401