[LON-CAPA-cvs] cvs: doc /packaging index.html
harris41
lon-capa-cvs@mail.lon-capa.org
Thu, 18 Jul 2002 00:30:42 -0000
This is a MIME encoded message
--harris411026952242
Content-Type: text/plain
harris41 Wed Jul 17 20:30:42 2002 EDT
Modified files:
/doc/packaging index.html
Log:
succinct-ifying and cool-ifying
--harris411026952242
Content-Type: text/plain
Content-Disposition: attachment; filename="harris41-20020717203042.txt"
Index: doc/packaging/index.html
diff -u doc/packaging/index.html:1.2 doc/packaging/index.html:1.3
--- doc/packaging/index.html:1.2 Tue May 23 12:33:28 2000
+++ doc/packaging/index.html Wed Jul 17 20:30:42 2002
@@ -1,321 +1,68 @@
-<HTML>
-<HEAD>
-<TITLE>LON-CAPA Packaging</TITLE>
-</HEAD>
-<BODY BGCOLOR=white>
-<H1>LON-CAPA Packaging and Filesystem Management</H1>
-<B>Functional Model Description</B> <I>(Scott Harrison, April 2000)</I>
-<P>
-<OL>
-<LI>Synopsis
-<LI>Summary of Data Store Objects <I>(e.g. CVS, RPM, LCIC)</I>
-<LI>Summary of Processes <I>(e.g. making a CD image, upgrading a system, altering package composition)</I>
-<LI>Summary of Actor Objects <I>(e.g. cron jobs, programmers, head packaging
-developer)</I>
-<LI>Functional Model Diagram
-<LI>Description of the Functional Model Diagram
-<OL>
-<LI>Data Store Objects
-<LI>Processes
-<LI>Actor Objects
-</OL>
-<LI>Frequently Asked Questions
-</OL>
-<P>
-<H1>1. Synopsis</H1>
-Up to this point, dozens of scripts have been written
-to automate various elements of the LON-CAPA <U>installation</U>, <U>upgrade</U>, and <U>software development</U> process. A number of significant issues
-relate to this effort. Security on the system is largely a matter of <B>limiting</B>
-unnecessary network services, <B>monitoring</B> the filesystem of LON-CAPA
-servers to make full account for the presence of every file on the system,
-and securely <B>communicating potential problems</B> to system administrators of a given
-LON-CAPA server machine.
-<P>
-This document is the functional model description which describes the computations
-and expected results necessary to manage packaging and filesystem management
-for LON-CAPA both now and in the future in a manner that is <B>synchronous</B>
-with software development, installations, and upgrades. Promulgation of coding
-changes, version control, and maintaining a handle on security and network optimization
-issues (such as introducing SAMBA and NetaTalk into the network architecture) have
-been some of the reasons for introducing a more strongly defined characterization
-of all of these processes involved with LON-CAPA software packaging and file system
-management.
-<P>
-<H1>2. Summary of Data Store Objects</H1>
-<UL>
-<LI><B>CVS</B>
-<BR>Concurrent Versions System. A central file repository is set up for LON-CAPA software development.
-<LI><B>RPM</B>
-<BR>Red Hat Package Manager. A system to manage software applications (packages). This system tracks package
-interdependencies, configuration files, and file alterations. The RPM packaging system can (and probably should) reside on a different
-computer than the central file repository computer. The RPM packaging system should probably be on a computer
-accessible to programmers involved with system development.
-<LI><B>LCIC</B>
-<BR>LON-CAPA Integrity Check. A script-based system which analyzes a computer to determine the RPM composition
-(missing or contrastingly extraneous RPMs), extraneous files which do not belong to any RPM, and RPM file
-alterations.
-</UL>
-<P>
-<H1>3. Summary of Processes</H1>
-<UL>
-<LI><B>Non-automated processes</B>
-<UL>
-<LI><B>implements change</B>
-<BR>Programmer creates a successful change to the LON-CAPA software. To carry out this step, the programmer
-must enter in bug-free code to the CVS file repository. The programmer must also inform the packaging developer
-as to where and how the files need to be placed and configured on LON-CAPA computer systems.
-<LI><B>alter package composition</B>
-<BR>The packaging developer may choose to add/remove different sets of entire RPM packages from LON-CAPA computer
-systems. The packaging developer may also choose to modify the file composition and dependency details of individual
-RPM packages. <I>There are over 700 potential RedHat packages to select from on RedHat 6.1 disks. Selecting an appropriate
-mix of RPMs can GREATLY improve security on LON-CAPA systems (by, for instance, removing unnecessary services) and is
-considered the general start- and end- point of LON-CAPA security.</I>
-<LI><B>change process programs</B>
-<BR>The various automated processes are to be written and modified by the packaging developer. These processes should
-reside on the packaging development system (RPM packaging system) as well the CVS file repository.
-<LI><B>upgrade</B>
-<BR>The packaging developer chooses when to upgrade the packaging system so that the packaging system upgrade passes
-along to other computers in the software development cluster.
-</UL>
-<LI><B>Automated processes</B>
-<UL>
-<LI><B>LON_CAPA_update_RPM_build_tree</B>
-<BR>Retrieves files out of the central software development CVS file repository. Builds any customized packages associated
-with the LON-CAPA system.
-<LI><B>LON_CAPA_make_CD_image</B>
-<BR>Compiles a RedHat installation/upgrade CD image based on the current LON-CAPA RPM packaging descriptions.
-<LI><B>LON_CAPA_post_CD_image</B>
-<BR>Make CD image available for other LON-CAPA development/run-time systems to upgrade from.
-<LI><B>LON_CAPA_upgrade_sys</B>
-<BR>Check to see if a general LON-CAPA RedHat system upgrade should be performed based on the version available from the packaging
-system. Also, monitor current status of system with LCIC (perhaps a "corrupt" system should be re-upgraded/re-installed). If upgrade
-should be performed, then perform a network install/upgrade while preserving all LON-CAPA configuration files.
-</UL>
-</UL>
-<P>
-<H1>4. Summary of Actor Objects</H1>
-<UL>
-<LI><B>Head Packaging Developer</B>
-<BR>Responsible for maintaining and writing scripts which automate the packaging and monitoring of distributed LON-CAPA systems. Also
-responsible for altering overall package composition of LON-CAPA systems.
-<LI><B>Programmer</B>
-<BR>Responsible for introducing software improvements to the LON-CAPA system itself. Submits software additions and modifications to
-the central CVS file repository.
-<LI><B>Cron</B>
-<BR>Cron jobs issue scheduled commands to facilitate the independent distributed processing of the various scripts associated with
-LON-CAPA Packaging and Filesystem Management.
-</UL>
-<P>
-<H1>5. Functional Model Diagram</H1>
-<IMG BORDER=1 SRC=scheme.gif>
-<P>
-<H1>6. Description of the Functional Model Diagram</H1>
-I will now describe a semi-chronological viewpoint of how all the different processes take place
-so as to better understand both the specifics of the functional model, as well as the pertinent
-dynamical and object-oriented characteristics of the algorithm.
-<P>
-Every week on Thursday at 3 in the morning, a <B>cron</B> job associated with the <B>RPM</B> packaging system on
-the main packaging computer updates the build tree of all LON-CAPA source RPMs based on files in the central
-<B>CVS</B> file repository. It does this by initiating the process <B>LON_CAPA_update_RPM_build_tree</B>. This process
-requests and receives files from the central <B>CVS</B> file repository, and updates all the source RPM file trees. RPM packages
-are then "bundled" by issuing build commands to the "Red Hat Package Manager" (<I>rpm -b ...</I>).
-<P>
-The <B>LON_CAPA_update_RPM_build_tree</B> process calls the <B>LON_CAPA_make_CD_image</B> process which writes a
-CD image into a single file on the main packaging system. This is followed by the process <B>LON_CAPA_post_CD_image</B>
-which mounts and posts the CD image file onto a web accessible location.
-<P>
-The <B>LON_CAPA_upgrade_sys</B> process is launched by a <B>cron</B> job specific to every individual
-computer present in the software development cluster. Each individual computer checks the main packaging computer to see
-if a system upgrade is needed. System upgrade information is passed to the <B>LCIC</B> on each specific computer (LON-CAPA
-Integrity Checker) so that integrity checking is done as a function of the most recent system upgrade.
-<P>
-Finally, on Sunday at 3 in the morning, a <B>cron</B> job associated with each computer launches the <B>LCIC</B> integrity checker
-system which resyncs itself based on the RPMs and files present on the system as well as information pertaining to
-the expected status of the system (in terms of upgrade, RPM composition, etc). The results of the <B>LCIC</B> integrity checker
-"re-syncing" are posted under a password controlled directory.
-<P>
-<H1>7. Frequently Asked Questions</H1>
-<P>
-<OL>
-<LI><B>What does LON-CAPA stand for?</B>
-<BR>The LearningOnline Network with a Computer-Assisted Personalized Approach
-<P>
-<LI><B>What is the general structure of the CVS repository?</B>
-<BR>These are the current directories:
-<BR><U>loncom</U> - server configuration and application files associated with LON-CAPA library and access servers
-<BR><U>lonline</U> - lecture online
-<BR><U>CAPA</U> - files associated with CAPA (homework processing, grading, homework generating, exam and quiz administration)
-<BR><U>doc</U> - documentation
-<BR><U>modules</U> - custom built software modules meant for use with the LON-CAPA project
-<BR><U>rat</U> - the "RAT"; resource assembly tool
-<BR><U>packaging</U> - files (including this one) specific to packaging and filesystem management of LON-CAPA
-<P>
-<LI><B>How does a programmer set himself up to use CVS?</B>
-<BR>
-<PRE>
-you will need to have the CVSROOT enviroment variable set to
-:pserver:yourname@zaphod.lite.msu.edu:/home/cvs
-To do this if you are using csh or tcsh put in your .login:
-setenv CVSROOT :pserver:yourname@zaphod.lite.msu.edu:/home/cvs
-For bash put this in .bash_profile
-export CVSROOT=:pserver:yourname@zaphod.lite.msu.edu:/home/cvs
-
-You can add the "cvs login" command to your .login/.bash_profile if you wish to make this simple.
-And you can add "cvs logout" to your .logout/.bash_logout so you rember to logout.
-
-To use CVS, you must first login (cvs login) and enter in your zaphod password.
-To leave CVS, you must logout (cvs logout).
-</PRE>
-<P>
-<LI><B>How does a programmer retrieve files from the CVS repository for the first time?</B>
-<BR><TT>cvs checkout [resourcename]</TT>
-<BR><TT>[resourcename]</TT> is one of the directories above (loncom, rat, doc, and so on)
-<P>
-<LI><B>How does a programmer update his files from the CVS repository?</B>
-<BR>(In other words, CVS server ----to----your---->> client computer.)
-<BR><TT>cvs update -d [resourcename]</TT>
-<BR><TT>[resourcename]</TT> is one of the directories above (loncom, rat, doc, and so on)
-<P>
-<LI><B>How does a programmer send his changes to the central CVS repository?</B>
-<BR>(In other words, client computer ----to----our---->> CVS server.)
-<BR><TT>cvs commit [resourcename]</TT>
-<BR><TT>[resourcename]</TT> is one of the directories above (loncom, rat, doc, and so on)
-<P>
-<LI><B>How does a programmer add or remove files from the central CVS repository?</B>
-<BR>Note: this is different from "updating" or "committing" changes. This is not the alteration
-of existing files. This is the ADDITION or REMOVAL of files.
-<BR>Within "the" directory, use these two commands to add a file:
-<BR><TT>cvs add [filename]</TT>; <TT>cvs commit</TT>
-<BR>Within "the" directory, use these two commands to remove a file:
-<BR><TT>cvs remove [filename]</TT>; <TT>cvs commit</TT>
-<P>
-<LI><B>How does a programmer add or remove directories from the central CVS repository?</B>
-<BR>Within the upper directory, use these two commands to add a directory:
-<BR><TT>cvs add [directoryname]</TT>; <TT>cvs commit</TT>
-<BR>Within "the" directory, use these two commands to remove a directory:
-<BR><TT>cvs remove [directoryname]</TT>; <TT>cvs commit</TT>
-<BR>There is a slight catch that you cannot directly "add" a new major directory like
-loncom, lonline, rat, doc, and so on. You must first "cvs add" it as a subdirectory, then
-move that subdirectory to the top directory structure and "cvs commit".
-<P>
-Better yet, you can <TT>cvs import</TT> your new directory into the top level area of the CVS repository.
-<LI><B>How does a programmer "inform" the packaging developer of changes to LON-CAPA system file composition?</B>
-<BR>By talking to the packaging developer in person.
-<BR>By making well-formatted changes to the <TT>packaging/file_map.txt</TT> file in the CVS repository. See below
-for more information on how to format the <TT>packaging/file_map.txt</TT> file.
-<P>
-<LI><B>What does "run any process" mean in the functional model?</B>
-<BR>Most processes are meant to run <I>automatically</I> and are invoked by cron jobs on various machines.
-It may be the case that, for instance, the processes need to be manually invoked so as to prepare
-a specific CD release.
-<P>
-<LI><B>Where are the automated processes located?</B>
-<BR>In the directory <TT>/usr/local/bin</TT>.
-<BR>By typing <TT>ls /usr/local/bin/LON_CAPA_*</TT>, you can view all the automated LON-CAPA processes associated with the computer systems.
-<P>
-<LI><B>Where are the <U>cron</U> jobs located? What are their names and how reliable are they?</B>
-<BR>On every LON-CAPA computer there is a file <TT>/etc/cron.d/LON_CAPA_filesystem</TT> which launches standard cron jobs.
-<BR>On the LON-CAPA main packaging computer there is a file <TT>/etc/cron.d/LON_CAPA_packager</TT> which launches cron jobs specific to the
-main packaging computer.
-<BR>On LON-CAPA software development computers, there is a file <TT>/etc/cron.d/LON_CAPA_upgrader</TT> which upgrades all computers in the
-LON-CAPA software development cluster.
-<P>
-<LI><B>Can automated processes be run manually? On the packaging computer? On other LON-CAPA computers?</B>
-<BR>Yes. To correctly manually run an automated process, look to see how the process is invoked in the <TT>/etc/cron.d</TT> directory.
-<P>
-<LI><B>What are the specifics associated with the RPM packaging system on the main packaging computer?</B>
-<BR>The RPM packaging system on the main packaging computer introduces the following differences to the main packaging computer compared to
-other computers in the software development cluster:
-<UL>
-<LI>Presence of RPM build tree
-<LI>Presence of CD image
-<LI>More automated processes present on system (LON_CAPA_update_RPM_build_tree, LON_CAPA_make_CD_image, LON_CAPA_post_CD_image)
-</UL>
-<P>
-<LI><B>What are the specifics associated with the process LON_CAPA_update_RPM_build_tree?</B>
-<BR>This process checkouts LON-CAPA directories and files. This process reads from the <TT>packaging/file_map.txt</TT> and the
-<TT>packaging/permissions.txt</TT> file. This process reads RPM <TT>.spec</TT> template files to set up the two LON-CAPA RPMs (LON-CAPA-base and LON-CAPA-auxiliary).
-These updated RPMs are placed into a pool of RPMs from which to generate the customized RedHat installation/upgrade CD.
-A 'comps' file is generated to customize the grouping of packages on the CD image. The CD image is written and made accessible
-on the web.
-<P>
-<LI><B>What are the specifics associated with the writing and web posting of the LON-CAPA system CD image on the main packaging computer?</B>
-<BR>There are a number of shell commands which need to be launched. (Details coming soon in this document.)
-<P>
-<LI><B>What are the specifics associated with the process LON_CAPA_make_CD_image?</B>
-<BR>There are a number of shell commands which need to be launched. (Details coming soon in this document.)
-<P>
-<LI><B>How are upgrades performed on the packaging computer? Is there any way to upgrade/install LON-CAPA on a new machine?</B>
-<BR>Upgrades are performed only based on package composition. New machines can be upgraded via network install or CD install. (More details coming soon.)
-<P>
-<LI><B>What are the specifics associated with the process LON_CAPA_post_CD_image?</B>
-<BR>There are a number of shell commands which need to be launched. (Details coming soon in this document.)
-<P>
-<LI><B>What are the specifics associated with the process LON_CAPA_upgrade_sys?</B>
-<BR>LON_CAPA_upgrade_sys does an automated network upgrade of any out-of-date packages based on the main packaging computer.
-LON_CAPA_upgrade_sys initializes and invokes the LCIC (integrity checker) based on current upgrade information.
-<P>
-<LI><B>What output from the LCIC (integrity checker) warrants taking corrective action?</B>
-<BR>Currently, the output from the LCIC consists of two files, TOTAL_FILE.txt and SUMMARY_FILE.txt. There are no immediate plans for
-automating any form of response to this output. This does provide information as to whether critical system binaries or configuration
-files are being tampered with. Perhaps the SUMMARY_FILE.txt should be e-mailed weekly to a system administrator at the responsible
-institution.
-<P>
-<LI><B>What is the correct format for the <TT>packaging/file_map.txt</TT> file?</B>
-<BR>Every line refers to a file or directory to install/upgrade on a LON-CAPA system. There are six fields to each line which are tab-separated.
-<PRE>
-[computer-type]
-[file location on CVS repository]
-[ultimate location on LON-CAPA system]
-[package name]
-[configuration settings that need to be saved?]
-[description of file and what it does]
-</PRE>
-<P>
-<LI><B>What is the correct format for the <TT>packaging/permissions.txt</TT> file?</B>
-<BR>Every line refers to a file or directory to install/upgrade on a LON-CAPA system. There are four/five fields to each line which are space-separated.
-File listing:
-<PRE>
-[octal mode]
-[user to own file]
-[group to own file]
-[file name]
-</PRE>
-Directory listing:
-<PRE>
-[octal mode]
-[user to own file]
-[group to own file]
-DIR
-[file name]
-</PRE>
-<UL>
-<LI><TT>[computer-type]</TT> - a list of capitalized letters indicating which computer-types this file is to be installed on. P=main packaging computer. A=LON-CAPA software development access server. L=LON-CAPA software development library server. C=central CVS file repository computer. PAL means this file is to be installed on the main packaging computer and the software development library and access servers. P means this file is to be installed on just the main packaging computer. AL means the file is to be installed on just the software development library and access servers.
-<LI><TT>[file location on CVS repository]</TT> - the file location on the CVS repository. For example, <TT>/loncom/htpasswd</TT>, <TT>loncom/startup.pl</TT>, and
-<TT>/rat/images/1.1.dt.gif</TT> are possible entries.
-<LI><TT>[ultimate location on LON-CAPA system]</TT> - where the file (taken from the CVS repository) is to be installed on the LON-CAPA system. For example,
-<TT>/home/httpd/lonTabs/htpasswd</TT>, <TT>/etc/httpd/conf/startup.pl</TT>, and <TT>/home/httpd/html/adm/rat/1.1.dt.gif</TT> are possible entries.
-<LI><TT>[package name]</TT> - What RPM package should this file be incorporated into for the final RPM install? Current possible entries are
-<TT>LON-CAPA-auxiliary</TT> and <TT>LON-CAPA-base</TT>.
-<LI><TT>[configuration settings that need to be saved?]</TT> - Every machine is anticipated to have a unique identity in terms of the role it
-fills at a given institution. For instance, /etc/httpd/conf/httpd.conf has settings like these:
-<PRE>
-PerlSetVar lonHostID msua3
-# Role of this machine: library, access
-PerlSetVar lonRole access
-# Server Administration
-PerlSetVar lonAdmEMail korte@lite.msu.edu
-# Default domain
-PerlSetVar lonDefDomain msu
-</PRE>
-At some point, these settings need to be saved and used during upgrade processes.
-<LI><TT>[description of file and what it does]</TT> - a brief sentence.
-</UL>
-<P>
-<LI><B>What controls version naming in terms of system upgrades?</B>
-<BR>This information is present in <TT>packaging/version.txt</TT>. This file consists of three tab-separated fields (version number, release number, and version
-name). The release number is automatically incremented each week as a function of LON_CAPA_post_CD_image. The version number and release numbers are used
-to label the LON-CAPA-base and LON-CAPA-auxiliary RPMs as well as the customized RedHat CD release.
-</OL>
-</BODY>
-</HTML>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html>
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
+<title>LON-CAPA Packaging</title>
+</head>
+<body bgcolor="white">
+<h1>LON-CAPA Packaging and Filesystem Management</h1>
+<b>Description</b> <i>(Scott Harrison, July 2002)</i>
+<ol>
+<li>Synopsis</li>
+<li>Diagram</li>
+</ol>
+<h2>Synopsis</h2>
+<table width="60%" border="1">
+<tr>
+<td bgcolor="#dddddd" width="33%"><font face="helvetica">Scene 1</font></td>
+<td bgcolor="#dddddd" width="33%"><font face="helvetica">Scene 2</font></td>
+<td bgcolor="#dddddd" width="33%"><font face="helvetica">Scene 3</font></td>
+</tr>
+<tr>
+<td width="33%">
+
+
+<img src="bug.gif" alt="a feature" /><br />
+<p>
+<strong>+ LON-CAPA<br />
+----------------<br />
+= LON-CAPA (new)</strong>
+</p>
+</td>
+<td>
+<p><strong>
+
+
+LON-CAPA (new)<br />
++ System Administrator<br />
+----------------<br />
+= </strong><img src="almostfixed.gif" alt="good times" /></p></td>
+<td>
+
+
+<img src="working.gif" alt="everything is wonderful" /></td>
+</tr>
+<tr>
+<td width="33%">
+<p>
+Once upon a time, a little green fellow added a "feature"
+to a hypothetical piece of software called LON-CAPA
+(for lack of a better term).
+</p>
+</td>
+<td width="33%">
+"Uh-oh" said the system administrator. "I have to upgrade
+my LON-CAPA system again?"
+</td>
+<td width="33%">
+But because the green fellow used XML to specify
+the software, there weren't any errors during the
+software upgrade, so everybody lived happily ever after.
+</td>
+</tr>
+</table>
+<h2>Diagram</h2>
+<img src="scheme.gif" alt="diagram of the scheme" />
+</body>
+</html>
--harris411026952242--