[LON-CAPA-admin] resources not updating on access servers

Thu Nov 20 11:14:24 EST 2008

Stuart,

The author (gonzales) has a directory under /home/httpd/lonUsers.  I'm
assuming it was created because I ran the make_domain_coordinator.pl
script during the install.  Is it OK to just delete the directories?

Bob Gonzales
Binghamton University
Chemistry Dept
-----Original Message-----
From: lon-capa-admin-admin at mail.lon-capa.org
[mailto:lon-capa-admin-admin at mail.lon-capa.org] On Behalf Of Stuart Raeburn
Sent: Thursday, November 20, 2008 6:50 AM
To: lon-capa-admin at mail.lon-capa.org
Subject: RE: [LON-CAPA-admin] resources not updating on access servers

Bob,

All three LON-CAPA DNS servers still have the following entries for  
your LON-CAPA servers in the binghamton domain:

binghamtonl1:binghamton:library:loncapa.chem.binghamton.edu
binghamtona2:binghamton:access:loncapa2.chem.binghamton.edu
binghamtona3:binghamton:access:loncapa3.chem.binghamton.edu

It does not appear that these entries are overridden by differences in  
the /home/httpd/lonTabs/hosts.tab om your library server.

The message: "Notifying host for metadata only  
binghamtona1:no_such_host" is, as Gerd suggested, an artifact of the  
fact that originally your first access server was identified as  
binghamtona1 in the hosts.tab files on your binghamton library server,  
before it was changed to binghamtona2 (for consistency with the entry  
in dns_hosts.tab that had been shared with the LON-CAPA DNS servers).   
These messages could be eliminated by editing (via command line) the  
meta.subscription and/or .subscription files in  
/home/httpd/html/res/$dom/$author/ for any problems housed on  
binghamtonl1, and accessed from binghamtona1 during that one week  
period, and removing lines such as

binghamtona1:128.226.130.9:1189617344

However, this is not necessarily recommended, as the current behavior,  
i.e., logging of a failure to notify binghamtona1 is correct since the  
lonHostID: binghamtona1 no longer exists in the network.

The "rejected" messages are more puzzling:

> Notifying host binghamtona2:rejected
> Notifying host binghamtona3:rejected

lond will return a result of "rejected" in response to an update to a  
resource to which a server is subscribed, if there is a user account  
for the author of the resource on the server which is being updated -  
the idea being that the copy in RES space on the library server which  
houses the author's construction space is automatically updated anyway  
on re-publication.

As both binghamtona2 and binghamtona3 are access servers it is not  
expected there will be anything contained in /home/httpd/lonUsers  
which could trigger the "rejected" response.  I would recommend  
checking the contents of /home/httpd/lonUsers on both  
loncapa2.chem.binghamton.edu and loncapa3.chem.binghamton.edu

You might also want to look at the entries in the .subscription file  
for this particular problem in /home/httpd/html/res/binghamton

Stuart Raeburn
MSU LON-CAPA group

Quoting Bob Gonzales <rgonzal at binghamton.edu>:

> So, now I get the following for the problem.log file for a recent update:
>
> Cleanup phase: Notifications
>
> Notifying host binghamtona2:rejected
> Notifying host binghamtona3:rejected
> Notifying host csma1:ok
> Notifying host for metadata only msua6:ok
> Notifying host for metadata only msua1:ok
> Notifying host for metadata only msua2:con_delayed
> Notifying host for metadata only msua5:ok
> Notifying host for metadata only msua3:ok
> Notifying host for metadata only msua4:con_delayed
> Notifying host for metadata only msua7:ok
> Notifying host for metadata only uiucaccess1:ok
> Notifying host for metadata only binghamtona1:no_such_host
> Notifying course binghamton_3c24739aca3844830binghamtonl1:ok
>
> There is still a binghamtona1 in there that doesn't exist but the
> binghamtona2 and binghamtona3 are both there and giving a
> 'rejected' response.
>
> The update was at 15:55, here's the lond.log from binghamtona3:
> Wed Nov 19 15:49:22 2008 (2623): <font color="green"> Attempting to start
> child (IO::Socket::INET=GLOB(0x1dfc6f0))</font>
> Wed Nov 19 15:49:22 2008 (22692): <font color="yellow">INFO: Connection,
> 127.0.0.1 (binghamtona3) connection type = client </font>
> Wed Nov 19 15:49:22 2008 (22692): Setting hostid to binghamtona3, and
domain
> to binghamton
> Wed Nov 19 15:49:22 2008 (22692): <font color="green"Successful local
> authentication </font>
> Wed Nov 19 15:49:22 2008 (22692): <font color='green'>Established
> connection: binghamtona3</font>
> Wed Nov 19 15:59:23 2008 (22692): Client 127.0.0.1 (binghamtona3) hanging
> up: exit:
> Wed Nov 19 15:59:23 2008 (22692): <font color='red'>CRITICAL: Disconnect
> from 127.0.0.1 (binghamtona3)</font>
> Wed Nov 19 15:59:23 2008 (2623): Child 22692 died
> Wed Nov 19 16:05:14 2008 (2623): <font color="green"> Attempting to start
> child (IO::Socket::INET=GLOB(0x1df3a40))</font>
> Wed Nov 19 16:05:14 2008 (23327): <font color="yellow">INFO: Connection,
> 127.0.0.1 (binghamtona3) connection type = client </font>
> Wed Nov 19 16:05:14 2008 (23327): Setting hostid to binghamtona3, and
domain
> to binghamton
> Wed Nov 19 16:05:14 2008 (23327): <font color="green"Successful local
> authentication </font>
> Wed Nov 19 16:05:14 2008 (23327): <font color='green'>Established
> connection: binghamtona3</font>
> Wed Nov 19 16:05:55 2008 (22670): Client 128.226.130.107 (binghamtonl1)
> hanging up: exit:
> Wed Nov 19 16:05:55 2008 (22670): <font color='red'>CRITICAL: Disconnect
> from 128.226.130.107 (binghamtonl1)</font>
>
> here's the lond.log from Binghamtona2
>
> Wed Nov 19 15:47:08 2008 (27610): Client 128.226.130.107 (binghamtonl1)
> hanging up: exit:
> Wed Nov 19 15:47:08 2008 (27610): <font color='red'>CRITICAL: Disconnect
> from 12 8.226.130.107 (binghamtonl1)</font>
> Wed Nov 19 15:47:08 2008 (2547): Child 27610 died
> Wed Nov 19 15:47:39 2008 (2547): <font color="green"> Attempting to start
> child(IO::Socket::INET=GLOB(0x201cd20))</font>
> Wed Nov 19 15:47:39 2008 (27949): <font color="yellow">INFO: Connection,
> 128.226.130.107 (binghamtonl1) connection type = client </font>
> Wed Nov 19 15:47:39 2008 (27949): Setting hostid to binghamtona2, and
domain
> to binghamton
> Wed Nov 19 15:47:39 2008 (27949): <font color="green">Successful insecure
> authentication with binghamtonl1 </font>
> Wed Nov 19 15:47:39 2008 (27949): <font color='green'>Established
> connection: binghamtonl1</font>
> Wed Nov 19 15:54:45 2008 (27769): Client 127.0.0.1 (binghamtona2) hanging
> up: exit:
> Wed Nov 19 15:54:45 2008 (27769): <font color='red'>CRITICAL: Disconnect
> from 127.0.0.1 (binghamtona2)</font>
> Wed Nov 19 15:54:45 2008 (2547): Child 27769 died
> Wed Nov 19 15:54:54 2008 (2547): <font color="green"> Attempting to start
> child (IO::Socket::INET=GLOB(0x2013760))</font>
> Wed Nov 19 15:54:54 2008 (28520): <font color="yellow">INFO: Connection,
> 127.0.0.1 (binghamtona2) connection type = client </font>
> Wed Nov 19 15:54:54 2008 (28520): Setting hostid to binghamtona2, and
domain
> to binghamton
> Wed Nov 19 15:54:54 2008 (28520): <font color="green"Successful local
> authentication </font>
> Wed Nov 19 15:54:54 2008 (28520): <font color='green'>Established
> connection: binghamtona2</font>
> Wed Nov 19 16:00:41 2008 (27949): Client 128.226.130.107 (binghamtonl1)
> hanging up: exit:
> Wed Nov 19 16:00:41 2008 (27949): <font color='red'>CRITICAL: Disconnect
> from 128.226.130.107 (binghamtonl1)</font>
>
> Any guesses on what is going on?
>
> thanks,
> Bob Gonzales
> Binghamton University
> Chemistry Dept
>
>
============================================================================
> ============================
> From: lon-capa-admin-admin at mail.lon-capa.org
> [mailto:lon-capa-admin-admin at mail.lon-capa.org] On Behalf Of Bob Gonzales
> Sent: Tuesday, November 18, 2008 11:49 AM
> To: lon-capa-admin at mail.lon-capa.org
> Subject: RE: [LON-CAPA-admin] resources not updating on access servers
>
> I followed the link mentioned below and republished a problem.  
> Here's the output from the problem.log file
>
> Cleanup phase: Notifications
> Notifying host msua7:ok
> Notifying host msua4:ok
> Notifying host msua6:ok
> Notifying host msua3:ok
> Notifying host msua5:ok
> Notifying host msua1:ok
> Notifying host msua2:ok
> Notifying host msua4:ok
> Notifying host msua2:not_found
> Notifying host msua3:not_found
> Notifying host msua6:not_found
> Notifying host authorl1:ok
> Notifying host msua7:not_found
> Notifying host msua5:not_found
> Notifying host msua1:not_found
> Notifying host authora1:ok
> Notifying host binghamtona1:no_such_host
> Notifying host csma1:ok
> Notifying host msua8:ok
> Notifying host binghamtona3:rejected
> Notifying host for metadata only uiucaccess1:ok
> Notifying host for metadata only fsul1:ok
> Notifying host for metadata only fsua2:ok
> Notifying course binghamton_9S24738c4a3844810binghamtonl1:ok
> Notifying course binghamton_1A2474087a384482fbinghamtonl1:ok
> Notifying course binghamton_3c24739aca3844830binghamtonl1:ok
> Notifying course binghamton_8R76021ee6d2048a6binghamtonl1:ok
> Notifying course binghamton_8315449109c4e4886binghamtonl1:ok
> ============ Done ============
>
> and lond.log on the loncapa3 access server.
>
> Tue Nov 18 10:32:21 2008 (2623): <font color="green"> Attempting to start
> child (IO::Socket::INET=GLOB(0x1dcb190))</font>
> Tue Nov 18 10:32:23 2008 (3854): <font color="yellow">INFO: Connection,
> 128.226.130.107 (binghamtonl1) connection type = client </font>
> Tue Nov 18 10:32:23 2008 (3854): Setting hostid to binghamtona3, and
domain
> to binghamton
> Tue Nov 18 10:32:23 2008 (3854): <font color="green">Successful insecure
> authentication with binghamtonl1 </font>
> Tue Nov 18 10:32:23 2008 (3854): <font color='green'>Established
connection:
> binghamtonl1</font>
> Tue Nov 18 10:36:41 2008 (3404): Client 127.0.0.1 (binghamtona3) hanging
up:
> exit:
> Tue Nov 18 10:36:41 2008 (3404): <font color='red'>CRITICAL: Disconnect
from
> 127.0.0.1 (binghamtona3)</font>
> Tue Nov 18 10:36:41 2008 (2623): Child 3404 died
> Tue Nov 18 10:36:44 2008 (2623): <font color="green"> Attempting to start
> child (IO::Socket::INET=GLOB(0x1cccfe0))</font>
> Tue Nov 18 10:36:46 2008 (4429): <font color="yellow">INFO: Connection,
> 127.0.0.1 (binghamtona3) connection type = client </font>
> Tue Nov 18 10:36:46 2008 (4429): Setting hostid to binghamtona3, and
domain
> to binghamton
> Tue Nov 18 10:36:46 2008 (4429): <font color="green"Successful local
> authentication </font>
> Tue Nov 18 10:36:46 2008 (4429): <font color='green'>Established
connection:
> binghamtona3</font>
>
> I assume the "Notifying host binghamtona1:no such host" is related to
Gerd's
> reply about the mis-naming of the
> server in the cluster hosts files.  The "Notifying host
> binghamtona3:rejected"   is the item of interest for
> the access server that is in the cluster hosts files. So, what does  
> the lond.log entries tell us?
>
> Also, regarding the binghamtona1/binghamtona2 server, did the hosts files
> get changed
> back to binghamtona1?  Things were working OK about a month ago.
>
> Bob Gonzales
> Binghamton University
> Chemistry Dept
>
> From: lon-capa-admin-admin at mail.lon-capa.org
> [mailto:lon-capa-admin-admin at mail.lon-capa.org] On Behalf Of H.K. Ng
> Sent: Tuesday, November 18, 2008 9:33 AM
> To: lon-capa-admin at mail.lon-capa.org
> Subject: RE: [LON-CAPA-admin] resources not updating on access servers
>
> I think I deleted everything under /home/httpd/html/res on the _access_
> server - just what you did. The access server will then get the most
> up-to-date version. Others may want to chime in on this.
>
> BTW, is your access server ever for a different purpose with different ip,
> etc. I vaguely remember when I re-configured a server, I miss a line in
the
> /etc/hosts file and this caused a number of problems.
>
> At 09:07 AM 11/18/2008, you wrote:
> HK,
>  There weren't any files in /home/httpd/sockets/delayed/.    How
> did you end up fixing your problem?
>  I was wondering if this action of not being updated could affect
> the viewing of grades and statistics.  That is to say, could I look at
> at a student's results on an access server that didn't update and
> see one set of grades and then look on the library server and see
> another newer set of grades?
>
> It does affect the total points for a sequence if a resource is on one
> server and not the other. Again, I had one of those (or similar) before!
See
>
>
http://mail.lon-capa.org/pipermail/lon-capa-admin/2006-November/001548.html
>
> -hk
>
>
>  Bob Gonzales
> Binghamton University
> Chemistry Dept
>  From: lon-capa-admin-admin at mail.lon-capa.org [
> mailto:lon-capa-admin-admin at mail.lon-capa.org] On Behalf Of H.K. Ng
> Sent: Tuesday, November 18, 2008 7:54 AM
> To: lon-capa-admin at mail.lon-capa.org
> Subject: Re: [LON-CAPA-admin] resources not updating on access servers
>  I had a similar problem a few years back and this is what Guy had to say,
>
> http://mail.lon-capa.org/pipermail/lon-capa-admin/2006-July/001407.html
>
> Maybe there is some files in the library server directory
> /home/httpd/sockets/delayed/ to indicate what
> happened.
>
>  -hk
>
>
> At 04:49 PM 11/17/2008, you wrote:
>
> I've got .problem resources that were re-published on my library server
but
> those updates were not propagated to my access servers.  It's been over a
> week.  I ended up deleting them on the access servers to force the update.
>
> How do I go about troubleshooting this?
>
> Thanks,
> Bob Gonzales
> Binghamton University
> Chemistry Dept
>
>
> _______________________________________________
> LON-CAPA-admin mailing list
> LON-CAPA-admin at mail.lon-capa.org
> http://mail.lon-capa.org/mailman/listinfo/lon-capa-admin
>
>
> _______________________________________________
> LON-CAPA-admin mailing list
> LON-CAPA-admin at mail.lon-capa.org
> http://mail.lon-capa.org/mailman/listinfo/lon-capa-admin
>

_______________________________________________
LON-CAPA-admin mailing list
LON-CAPA-admin at mail.lon-capa.org
http://mail.lon-capa.org/mailman/listinfo/lon-capa-admin