[LON-CAPA-admin] Not Open To Be Viewed Troubles

[Raeburn, Stuart]

Lee,

The two additional types of behavior you report relate to resources for which a timed (interval) parameter applies.  In such a case, determining whether the resource will be available to a particular student for making submissions depends on parameters set for the course (or section), i.e., data from resourcedata.db for the course (cached on an access server for 10 minutes), and also data stored for the specific user, i.e., data from firstaccesstimes.db for the user (on the library server).

When a student accesses a resource for which a timed (interval) parameter applies, the scope of the interval in effect can be resource, map or course. If no firstaccess time is retrieved for a student for the appropriate identifier for that scope, then the status returned by lonhomework::check_access() will be "NOT_YET_VIEWED", assuming other parameters -- opendate, duedate, answerdate or acc -- or slot control don't prevent availability.

If status is "NOT_YET_VIEWED" then the "Show Resource" button will be available for the student to start the timer.
If the button is pushed, but the firstaccess time is not recorded on the user's home server (i.e., your library server), then the page sent back to the browser will again contain the "Show Resource" button.  Repeated pushes of the button (with no recording of the data) will result in repeated markaccess entries in the student's activity log.  

Note: if the result of the "put" for storage of the firstaccess time is "refused", then an item will be logged in /home/httpd/perl/logs/lonnet.log. If the failure is for some other reason -- e.g., lond::put_user_profile_entry() could not tie the student's firstaccesstimes hash or there was a con_lost response, then nothing will be logged in lonnet.log.

As regards the report from a student of receiving additional time midway through an exam, one legitimate way that could happen is if a Course Coordinator used: Settings > Content Settings >  "Reset Student Access Times" to reset that particular student's access time during the exam.

Another way a student could receive additional time would be if the call to &EXT('resource.0.interval') when accessing a resource in the exam folder did not retrieve the interval setting (e.g., because of a caching issue or a problem with data retrieval from the library server when populating the cache). Note: in this case this would be expected to impact other students as well.  In the absence of an interval parameter, the time available will be determined from the due date in effect.

A further way a student could receive additional time (just for himself/herself) would be if the previously set firstaccess time (set by the student, and which applied to the exam folder) was not retrieved when accessing a resource in the folder (and the scope of the time limit was the folder). In that instance, the "Show Resource" button would be displayed, and if the button were pushed, a new (later) firstaccess time would be set, if the previously set firstaccess time could not be retrieved a second time (this time on a different request), when processing the markaccess submission.  The /home/httpd/lonUsers/uiuc/<1>/<2>/<3>/<username>/firstaccesstimes.hist file for the student (where <1>, <2>, and <3> are the first three characters of the username) would contain a record of all stored firstaccess times.

One way for LON-CAPA to be more robust against intermittent problems with retrieval of firstaccesstimes from a library server, would be to make use of $env{'course.'.$courseid.'.firstaccess.'.$res}, which is stored in the user's session file on the access server. Any existing firstaccess times for a user are added to %env as part of the user session initialization following log-in. New items of this type will be added to %env whenever a firstaccess time is successfully sent to the library server after the "Show Resource" button had been pushed.

Note: if use were made of these firstaccess items in %env in place of (or in addition to) the current call to lonnet::dump() in lonnet::load_all_first_access(), then once a Course Coordinator used "Reset Student Access Times", a student affected by that would need to logout of an active session, and log-in again to see th change. (The alternative would be to identify all nodes with active session files for an affected user, and delete any "course.'.$courseid.'.firstaccess.'.$res" items from each session file, when executing the reset for the particular $res context).

When you say you removed the local files as a precaution, do you mean you removed the replicated resource files and/or replicated metadata files for the resource?  The message which would be displayed to a user if a resource file was missing from the access server (and could not be replicated) would be: Unable to find <filename>.  I have not seen that error reported in any of your posts to this "Not Open To Be Viewed Troubles" thread over the past 18 months when you have seen these occasional problems with unavailability of resources.  The premise has been that the issue you have seen has been with parameters controlling availability of the resource to the student, and not with access to the content of the file itself on the filesystem. Accordingly, I would not expect removal of copies of the resource(s) from an access server (which will force replication of the content) would have any bearing on the issue on a "Not Open To Be Viewed" problem.  One case where removing files might influence this would be if the resource's metadata file itself contained the parameters controlling availability (and you'd removed the .metadata file).  It would be unusual to include a parameter such as interval in a resource itself, but there might be use cases where it makes sense to do that (e.g., see: mail.lon-capa.org/pipermail/lon-capa-users/2018-September/005351.html prior to the availability of recursive parameters).

But perhaps by "local files" you mean something else, such as a user's course session files in /home/httpd/perl/tmp, removal of which would force a user with an expired session to re-select the course.

I have just committed lonnet.pm rev. 1.1386 which includes changes so failure to store a firstaccess time will be logged in more cases.  See: mail.lon-capa.org/pipermail/lon-capa-cvs/Week-of-Mon-20181015/028316.html
I'll also make a change to structuretags.pm so a message will be displayed on screen displayed after the "Show Resource" button was pushed (and submitted), if the firstaccess time could not be stored.

Stuart Raeburn
LON-CAPA Academic Consortium

________________________________________
From: LON-CAPA-admin <lon-capa-admin-bounces at mail.lon-capa.org> on behalf of Bynum, Lee Hamilton <leebynum at illinois.edu>
Sent: Tuesday, October 16, 2018 3:13:47 PM
To: list about administration and system updating
Subject: Re: [LON-CAPA-admin] Not Open To Be Viewed Troubles

I have another batch of these unavailable resources.  While I dig into the new data points, I figure I should give an update.

It is our suspicion that removing the local files from the access server helps prevent this from occurring.  We had an incident on Friday on a single access server.  We removed the local files on that server as a precaution.  Over the weekend, more incidents occurred, but not on the access server we cleared the files from.

During the same period we had two additional behaviors that seem related to access server files not syncing with the library properly.  One student reported being given additional time on an exam midway through.  This behavior is less likely to be reported, so it is unclear if there are more incidents.  There is also a behavior in which a student repeatedly needed to log in to answer the same questions over again.  This is reflected in the log as an answer submitted but not evaluated as the access was removed.  The odd part of that is that it results in multiple markaccess entries.

With this newest batch we were able to demonstrate this behavior isolated to individual students despite other students using the same resources at the same time on the same server.

It is also worth noting that this is the first batch of incidents this semester.  It is unclear what might have changed on our servers to prompt it, but that is definitely something we are looking into.

In short, suddenly unavailable resources continue to pop up and we are still investigating.

Lee

> -----Original Message-----
> From: lon-capa-admin-bounces at mail.lon-capa.org <lon-capa-admin-
> bounces at mail.lon-capa.org> On Behalf Of Bynum, Lee Hamilton
> Sent: Wednesday, February 21, 2018 9:50 AM
> To: list about administration and system updating <lon-capa-admin at mail.lon-
> capa.org>
> Subject: Re: [LON-CAPA-admin] Not Open To Be Viewed Troubles
>
> Stuart,
>
> This has been very helpful for digging into what's going on.  In case I'm barking
> up the wrong tree, though, I'd like to sanity check some potential state values.
> I'm seeing values that are either single digit integers or a series of integers,
> which I suspect come from the sequence based classes.  For example, "2" or
> "2002202".  Are those the sort of values to be expected in user.state?
>
> Lee
> ________________________________________
> From: lon-capa-admin-bounces at mail.lon-capa.org [lon-capa-admin-
> bounces at mail.lon-capa.org] on behalf of Stuart Raeburn [raeburn at msu.edu]
> Sent: Wednesday, October 04, 2017 10:28 PM
> To: lon-capa-admin at mail.lon-capa.org
> Subject: Re: [LON-CAPA-admin] Not Open To Be Viewed Troubles
>
> Lee,
>
> Reviewing my previous answer, I find that I provided an answer to a different
> question than the one you had actually asked.
>
> Accordingly, I am going to revise my answer from: "it is in memory on the access
> server", to it is in memory (in %env) and also in the user's session file, on the
> filesystem on the access server.
>
> As you noted the username_courseid.state file in /home/httpd/perl/tmp
> contains the conditional statement:
>
> &EXT('user.resource.resource.200.1.awarddetail','BuffersLab') eq 'APPROX_ANS'
>
> The user's session file (a GDBM file):
>
> /home/httpd/lonIDs/username_sessionnum_domain_authhost.id
>
> will contain a key of: user.state.uiuc_3g9196182397259fcuiuclibrary1,
> with the corresponding value containing the results of evaluated conditions.
>
> The routine: &Apache::lonuserstate::evalstate() is used to determine (and save)
> the value for:
> user.state.uiuc_3g9196182397259fcuiuclibrary1, which is appended to %env,
> and also to the session file.
>
> The evalstate() routine will evaluate the conditions in perl Safe space, which will
> include calling lonnet::EXT(), which in turn calls lonnet::restore to retrieve the
> user's submission history for the resource to which the alias: 'BuffersLab' points.
>
> The routine: &Apache::lonnet::devalidate() will remove
> user.state.uiuc_3g9196182397259fcuiuclibrary1 from %env (and from the user's
> session file).
>
> The devalidate() routine is called whenever lonnet::store() or lonnet::cstore are
> called in course context to store the user's submissions, points etc.
>
> Note: the question I answered with my previous response to this thread was
> where can you find values for items such as:
> &EXT('user.resource.resource.200.1.awarddetail','BuffersLab') when a problem
> is being rendered, and in that case the answer is in memory.
>
> Stuart Raeburn
> LON-CAPA Academic Consortium
>
> >>
> >> Does this live entirely in memory or am I missing something?
> >>
> >
> > It is in memory on the access server.
> >
> > When the problem is rendered, the &initialize_storage() routine is
> > called in structuretags.pm within start_problem() which will
> populate
> > the (global) %history hash in Apache::lonhomework with submission
> and
> > award history etc. for the current resource (in the current course
> > context and instance for the current user) by calling
> > lonnet::restore().
> >
> > See lines 1024-1025 in the version of structuretags.pm included in
> > loncapa-2.11.2.
> >
> > Later during rendering, the global %history hash is cleared with undef
> > (within the end_problem routine).
> >
> >
> > Stuart Raeburn
> > LON-CAPA Academic Consortium
> >
> >> I'm tracing the path of where a condition is stored against the
> > files
> >> on our servers.
> >>
> >> Condition to be met:
> >>
> > &EXT('user.resource.resource.200.1.awarddetail','BuffersLab') eq
> >> 'APPROX_ANS'
> >>
> >>
> >
> Library1:/home/httpd/lonUsers/uiuc/u/s/e/username/uiuc_3g9196182397259fc
> uiuclibrary1.db
> >
> >> contains:
> >>
> >
> 2:uiuc/dmills/CHEM105/Buffers/Buffers.sequence___5___uiuc/dmills/CHEM105
> /Buffers/Lab.problem:resource.200.1.awarddetail
> > =
> >> APPROX_ANS
> >>
> >> I'm having a difficult time locating that condition on the access
> servers, though.  I've found the conditional statement in the .state
> >
> >> file, but I'm not seeing it in any of the user files.  Does this
> > live
> >> entirely in memory or am I missing something?
> >>
> >> Thanks,
> >>
> >> Lee
> >>
> >>
> >> -----Original Message-----
> >> From: lon-capa-admin-bounces at mail.lon-capa.org
> >> [mailto:lon-capa-admin-bounces at mail.lon-capa.org] On Behalf Of
> > Stuart
> >> Raeburn
> >> Sent: Tuesday, October 3, 2017 7:14 PM
> >> To: lon-capa-admin at mail.lon-capa.org
> >> Subject: Re: [LON-CAPA-admin] Not Open To Be Viewed Troubles
> >>
> >> Hello Lee,
> >>
> >> There is no log file of access server parameter file changes, but
> > you
> >> could make your own (see below).
> >>
> >> Looking back through the archives, it seems you ran into this issue
> > in March.
> >>
> >> I posted a response to the list to that earlier post. See:
> >>
> >> mail.lon-capa.org/pipermail/lon-capa-admin/2017-March/003273.html
> >>
> >> which discussed how to view cached parameters in memcache.
> >>
> >> Assuming the open date parameter in effect is for all students, or
> >> for a specific section, and not for individual student(s), then
> >> parameters will be retrieved using lonnet::get_courseresdata().
> That
> >
> >> will, in turn, call &lonnet::dump(), when the cache on the access
> >> server has expired (it's valid for 10 minutes).
> >>
> >> You could modify lonnet::get_courseresdata() to write the hash
> >> returned by dump() for a particular course to a file (e.g.,
> >> /home/httpd/perl/tmp/debug/$coursenum on the access server).
> >>
> >> Your code could check if the file already exists, and if it did
> move
> >
> >> the existing file to /home/httpd/perl/tmp/debug/$coursenum.old,
> >> before writing the latest data to a new file.
> >>
> >> You could then create a perl script, which would be run by cron
> > every
> >> 5 minutes to look in /home/httpd/perl/tmp/debug/.  The script would
> compare the contents of $coursenum.old and $coursenum (if both
> > exist)
> >> and record what had changed for course: $coursenum for the current
> timestamp, in a log file.  The script would then unlink $coursenum.old.
> >>
> >>
> >> Stuart Raeburn
> >> LON-CAPA Academic Consortium
> >>
> >>
> >>> Hello Admins,
> >>>
> >>> I am diving back into an issue we've had in which resources are
> >>> temporarily "Not open to be viewed."  The resources should be
> >>> available now and do become available after 10-20 minutes or a
> >> change
> >>> of access server.  The leading theory is that the parameter file
> is
> >>> being read incorrectly, resulting in the access server thinking
> the
> >>> resource is unavailable until the bad parameter is updated.
> >>>
> >>> Is there a log anywhere of access server parameter file changes?
> >>> Because things seem to go back to normal after a while I have yet
> > to
> >>
> >>> have the problem reported to me in time to look at the potentially
> >>> problematic files in real time.
> >>>
> >>> Sorry for rehashing old troubles, but I'm hoping this time will to
> >> the trick.
> >>>
> >>> Lee
> >>
> >> _______________________________________________
> >> LON-CAPA-admin mailing list
> >> LON-CAPA-admin at mail.lon-capa.org
> >
> > _______________________________________________
> > LON-CAPA-admin mailing list
> > LON-CAPA-admin at mail.lon-capa.org
>
> _______________________________________________
> LON-CAPA-admin mailing list
> LON-CAPA-admin at mail.lon-capa.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.lon-2Dcapa.org_mailman_listinfo_lon-2Dcapa-2Dadmin&d=DwIFAw&c=nE__W8dFE-shTxStwXtp0A&r=VsGo3jOm8tGLd6f-KlhT-g&m=yj7qi3vKIoDz_xz3y7eeFLj7D01cnv9o-FO832PpAoI&s=BDVMAsSGWOL_NBzr7NMUlJ6v0V2anyeIZtI3vQBDCTo&e=
> _______________________________________________
> LON-CAPA-admin mailing list
> LON-CAPA-admin at mail.lon-capa.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.lon-2Dcapa.org_mailman_listinfo_lon-2Dcapa-2Dadmin&d=DwIFAw&c=nE__W8dFE-shTxStwXtp0A&r=VsGo3jOm8tGLd6f-KlhT-g&m=yj7qi3vKIoDz_xz3y7eeFLj7D01cnv9o-FO832PpAoI&s=BDVMAsSGWOL_NBzr7NMUlJ6v0V2anyeIZtI3vQBDCTo&e=
_______________________________________________
LON-CAPA-admin mailing list
LON-CAPA-admin at mail.lon-capa.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.lon-2Dcapa.org_mailman_listinfo_lon-2Dcapa-2Dadmin&d=DwIFAw&c=nE__W8dFE-shTxStwXtp0A&r=VsGo3jOm8tGLd6f-KlhT-g&m=yj7qi3vKIoDz_xz3y7eeFLj7D01cnv9o-FO832PpAoI&s=BDVMAsSGWOL_NBzr7NMUlJ6v0V2anyeIZtI3vQBDCTo&e=