[LON-CAPA-admin] Problems during Course Initialization occur after cloning a course

[Stuart Raeburn]

Hello Martin,

> ... Do you have any hint for realizing that or is disabling
> the certificate check the only way to keep it running at the moment?

The reason you were seeing this error before disabling verification with
PERL_LWP_SSL_VERIFY_HOSTNAME = 0 added to /etc/environment was an  
inconsistency between the hostname being included for your server in  
LON-CAPA replication requests and the common name included in the SSL  
certificate.

You can fix this issue by modifying the hostname for your server in  
/home/httpd/lonTabs/hosts.tab and restarting the LON-CAPA daemons.

sudo /etc/init.d/loncontrol restart

Replace:

tucl1:tuc:library:gaia.rz.tu-clausthal.de:https:tu-clausthal.de

with

tucl1:tuc:library:loncapa.tu-clausthal.de:https:tu-clausthal.de

Once you have done that you will also be able to eliminate your  
workaround modification to /etc/environment.  You might also need to  
modify /etc/hosts if you included hostname to IP mapping, i.e., in  
that file you might need:

139.174.32.37 loncapa.tu-clausthal.de loncapa

*****

Additional information:

The common name for your certificate is: loncapa.tu-clausthal.de

but your /home/httpd/lonTabs/hosts.tab identifies your server as:

gaia.rz.tu-clausthal.de:443

Although, loncapa.tu-clausthal.de is an alias to  
gaia.rz.tu-clausthal.de, your SSL certificate is for a single domain,  
so gaia.rz.tu-clausthal.de is not being verified when LWP::UserAgent()  
makes file requests.

Because of this inconsistency other LON-CAPA domains running LWP 6.0  
and later (i.e., Ubuntu 12, 14 and CentOS/RedHat/Scientific Linux 7)  
are unable to replicate content from the tuc domain.

> I tried using wget and got a certificate error (19, self signed). But I
> could not find the reason as the certificate chain is configured
> correctly

You can check whether a certificate chain is correctly configured using:
openssl s_client -connect loncapa.tu-clausthal.de:443 -showcerts

You will see a "19, self-signed" unless you include the arg: -CApath  
to point at the directory containing trusted root certificates.

On an Ubuntu system you can keep root certificates up to date by  
installing the ca-certificates package:

sudo apt-get install ca-certificates

The root certificates will be installed in /etc/ssl/certs

So the command:

openssl s_client -connect loncapa.tu-clausthal.de:443 -CApath /etc/ssl/certs

should report no errors for your certificate chain.

Similarly with wget you might need to explicitly point at the location  
of the trusted root certificates:

wget https://loncapa.tu-clausthal.de/ --ca-directory=/etc/ssl/certs

to avoid a "(19, self signed)" certificate error (i.e., a false positive).


Stuart Raeburn
LON-CAPA Academic Consortium


Quoting Martin Diedrich <diedrich at rz.tu-clausthal.de>:

> Hello Stuart,
>
> many thanks for pointing me to that issue!
>
> I tried using wget and got a certificate error (19, self signed). But I
> could not find the reason as the certificate chain is configured
> correctly and there were no failures using firefox browser connecting
> to the machine. The certificate itself is signed by a CA and no
> certificate in the chain is self-signed except for the root CA
> certificate. Using wget from other machines connecting to our
> loncapa-server the problem sometimes occured, sometimes it didn't -
> depending on the machine used (not indeterministic by request or time).
>
> I found out that there is an environment variable existing to disable
> verification of certificates (PERL_LWP_SSL_VERIFY_HOSTNAME = 0). When I
> added this to /etc/environment, everything worked fine as far as I
> could check yet. This is ok as it's the behavior from before the Ubuntu
> update but I'd surely prefer checking the certificates as it's done by
> default today. Do you have any hint for realizing that or is disabling
> the certificate check the only way to keep it running at the moment?
>
> As I said before - many thanks for your quick and competent responses
> in this list, this is for sure very very helpful :)!
>
> Greetings from germany
> Martin Diedrich
>
> Am 23.09.2014 18:00, schrieb Stuart Raeburn:
>
>> Martin,
>>
>>> Doing a test by cloning the original course again results in getting
>>> the same error. No other courses seem to be affected.
>>
>>> Invalid map: /home/httpd/html/adm/notfound.html
>>
>> This error means that the initial replication of the default.sequence
>> file (the XML file of course contents at the top level folder in the
>> course) failed, resulting in a top-level course URL of
>> /adm/notfound.html.
>>
>> I would however expect that cloning etc. would have proceeded despite this.
>>
>> The most likely reason for the replication failure is that your Apache
>> default-ssl.conf file does not include an entry for the correct Server
>> Certificate Chain.
>>
>> Look for: SSLCertificateChainFile in that file.
>>
>> The version of libwww-perl used by Ubuntu 12.04 will not, by default,
>> retrieve a file from an https:// URL unless the certificate chain can
>> be used to verify the hostname.
>>
>> If it cannot, LWP will return a 500 error, and the URL for the course
>> will be set to /adm/notfound.html
>>
>> Look in /home/httpd/perl/logs/lonnet.log for:
>>
>> <font color="blue">WARNING: LWP get: 500
>> Can't connect to <hostname>:443 (certificate verify failed):
>>
>> You could also check in /var/log/apache2/ssl_access_log for entries
>> similar to:
>>
>> "GET /userfiles/tuc/<courseid>/default.sequence HTTP/1.1"
>>
>> where <courseid> is the internal courseID of the course you created in
>> the tuc domain.
>>
>> However, if certificate verification did indeed fail, LWP will not
>> have made the request, so you'll not find a "GET" for that URL logged
>> in ssl_access_log.
>>
>>  From the command line you might check whether the certificate chain
>> is valid by using:
>>
>> wget   
>> https://loncapa.tu-clausthal.de/userfiles/tuc/<courseid>/default.sequence
>>
>> replacing <courseid> with the internal courseID of the course.
>>
>>
>> Stuart Raeburn
>> LON-CAPA Academic Consortium
>
> -- 
> Dipl.-Wirt.-Inf. Martin Diedrich   E-Mail: diedrich at rz.tu-clausthal.de
> Rechenzentrum TU Clausthal         Telefon: +49 5323 72 2522
> Erzstrasse 51                      Jabber: ifmdi at jabber.tu-clausthal.de
> 38678 Clausthal-Zellerfeld         WWW: http://www.rz.tu-clausthal.de/