Add SYSHST documentation files.

Makefile

@@ -8,7 +8,7 @@ SRC = system syseng sysen1 sysen2 sysen3 sysnet kshack dragon channa	\
       midas _teco_ emacs emacs1 rms klh syshst sra mrc ksc eak gren	\
       bawden _mail_ l lisp libdoc comlap lspsrc nilcom rwk	\
       inquir acount gz sys decsys ecc alan sail kcc kcc_sy c
-DOC = info _info_ sysdoc sysnet kshack _teco_ emacs emacs1 c kcc
+DOC = info _info_ sysdoc sysnet syshst kshack _teco_ emacs emacs1 c kcc
 BIN = sys2 device emacs _teco_ lisp liblsp alan inquir sail comlap c
 
 # These directories are put on the minsys tape.

doc/syshst/-read-.-this-

@@ -0,0 +1,19 @@
+AI: SYSHST; directory for host tables.
+
+This is the home of the master sources for the MIT host tables,
+or for copies in cases where the master source is a LispM namespace
+or some such.  Changes to files should be announced to INFO-HOSTS@AI,
+bug reports as well.  See -WHAT- -FILE- for more info.
+
+XX:<HOSTS> (available as XX: HOSTS; from ITS EMACS) is a backup of
+this directory and contains various other cruft.  There's a nightly
+batch job on XX to keep things current.
+
+Please do not touch anything in this directory unless you know what
+you are doing.  Read the documentation and ask the people on
+INFO-HOSTS if it's not clear.
+
+There is some documentation of XXRFC810 format in -THIS- -TOO-.
+There is an explaination of which files are what in -WHAT- -FILE-.
+
+--SRA 7 October 86.

doc/syshst/-this-.-too-

@@ -0,0 +1,155 @@
+This file describes the state things were in around December 1985.
+Since then I have moved the host compilation process for ITS to XX
+(ie, XX now compiles tables and installs them on the ITS machines as
+well as on XX itself).  The ITS flavors of the various TECO macros,
+XFILEs, etc, are preserved in AI: SYSHST; AR2 HSTSRC.  They need to be
+kept around as a backup in case XX is down for long periods of time.
+
+If you want to figre out what is going on these days, browse through
+AI: SYSHST; and XX: HOSTS;.
+
+--sra 17 June 86
+----------------------------------------------------------------
+
+[Bugs in MAKHST, MAKDOM, or this documentation to SRA@XX]
+
+This file describes the new host table format (or "extended extended
+RFC810 format").  This is the format used to store the host/domain
+data for all MIT machines (or that is the intent anyway).  The format
+is an extension of the format used by the HOSTS3 compiler ("extended
+RFC810 format").  The basic motivation for all this cruft is to keep
+all the host data in a single set of files, in spite of the fact that
+this data is used in entirely different ways by the Chaosnet and the
+Internet.  If you want to flame about the religious issues involved
+here, send mail to NAMECALLERS@MC, a list of people who are
+collectively responsible for implementing this crock.
+
+== Format:
+
+The only real change is the addition of a new entry type, DOMAIN.
+This is still somewhat freeform; I am open to suggestions as to
+improvements.  The DOMAIN entry contains three kinds of information,
+all related to the domain system (but read on, this matters to
+Chaosnet people too).  These are: (1) the name of the domain (loading
+origin if this is fed into a nameserver), (2) the data for the SOA RR,
+and a list of nameservers (with optional addresses) for this domain.
+  The format is:
+
+DOMAIN : <domain-name> : <class> : <source>: <bug-address> : 
+ 	 <serial> : <refresh> : <retry> : <expire> : <minimum> :
+ 	 <nameserver> {<address>} {,<nameserver> {<address>}} :
+ 
+(where all of this is on a single line, of course).  Of these, only
+the <domain-name> field is currently of any consequence to Chaosnet
+hosts or to hosts that keep their SOA and NS data in a seperate file.
+Thus the brief form looks like
+
+DOMAIN : EECS.MIT.EDU :
+
+This will work if fed to the MAKHST program, which doesn't care about
+SOAs or NSs.  It will *not* work if fed to MAKDOM.  People maintaining
+domain nameservers presumably understand what these fields mean (see
+RFC883 if you aren't sure).  An example of the complex form (again,
+suppressing the line break):
+
+DOMAIN : LCS.MIT.EDU : IN : XX.LCS.MIT.EDU : BUG-LCS-DOMAIN.XX.LCS.MIT.EDU
+       : 0 : 7200 : 600 : 3600000 : 60 : XX.LCS.MIT.EDU 10.0.0.44,
+         MILO.LCS.MIT.EDU :  
+
+Things to note:
+
+The <address> fields may be blank if the machines acting as
+nameservers are all in this domain (and thus have their addresses
+listed elsewhere in this file).  If there is an address it is the
+entire rest of the subfield (up to the delimiting comma or colon);
+this allows for future expansion for Chaosnet nameservers if these
+prove to be useful.
+
+The <serial> field may be zero; in this case the <serial> value in the
+SOA record is the version number of the source file containing this
+DOMAIN entry.  Obviously this only makes sense when the master copy of
+the file is kept on a machine that implements version numbers.
+
+The existing filter programs assume that the DOMAIN entry, if it
+exists at all, will come before any HOST, GATEWAY, or NET entries.
+This could be fixed if anybody gives me a good reason.
+
+One other note: the redundant nicknames for machines have been
+removed.  Ie, where there used to be an entry listing names
+"MIT-XX.ARPA, MIT-XX, XX", there will now be an entry that only lists
+"XX".  The filters can cons up this kind of trivial name easily, and
+it turned out to be a lot easier to add the "MIT-"s and ".ARPA"s in
+the right places than to remove them from the wrong ones.
+
+== Programs:
+
+There are currently two filter programs, MAKDOM and MAKHST.  Both
+could use improvement if anybody is so inclined.
+
+
+MAKDOM:
+
+This is a C program which will run on Twenex or Berkeley Unix.  Since
+these are currently the only operating systems likely to be running
+domain nameservers (as opposed to resolvers), this seemed like a
+reasonable choice.  It converts this host table format into the form
+specified in RFC883 for zone master files.  It behaves like a vanilla
+unix filter program, so you can pipe to it, etc, if you have some
+reason to do so.  Documentation at this point is just a long comment
+at the begining.  There are various switches to control what kind of
+RRs are written and to control how many forms of generated nicknames
+to add, etc.  Ask me (sra@xx) if you want a copy.
+
+
+MAKHST:
+
+This is a filter to convert the new host table format into something
+that can be fed to the HOSTS3 compiler (or used directly by any
+machine that has been using MC: SYSNET; HSTMIT >).  It is implemented
+as (gasp) a dumped TECO program.  It has a semi-reasonable command
+scanner which will parse a JCL that looks similar to most old ITS or
+BOTTOMS-10 programs (if you run it without any JCL it will type out an
+error message explaining the argument format).  Since HOSTS3 only runs
+on PDP10s and the host tables all currently live on AI (with copies on
+XX), the fact that TECO only runs on PDP10s should not be much of a
+restriction.  There are (or will be) XFILEs and .CTL files to do the
+right frobs to run this filter and feed the results to HOSTS3.  Like
+MAKDOM, MAKHST has various switches that let you control nickname
+hacking, DOMAIN entry parsing, etc.
+
+There is a third filter program, HSTNIC.  This is also written in
+TECO.  It is a little different from the others in that it is not
+concerned with the new host table format.  Rather, it does two things.
+Firstly, it removes all references to MIT from HSTNIC, since all MIT
+hosts should now be in our own tables, quite possibly with different
+names than the NIC has on file.  Secondly, HSTNIC punts any machine
+names in the HSTNIC file that would conflict with MIT nicknames.  Eg,
+UW-EDDIE's nickname of EDDIE gets punted because that conflicts with
+the MIT machine in building 38.  The current implementation of this is
+extremely kludgy and takes much longer than it should.  I will fix it
+some day if I get around to it....
+
+== Files:
+
+HSTMIT has been split up into a number of files, one per domain within
+MIT and a few extras for groups within MIT that will probably become
+domains soon.  The reason for the split is that the source files will
+probably not all live on the same machine any more.  (There have to be
+copies on whatever machine is putting together HOSTS3 files, but these
+will probably be secondary copies that are kept current via FTP.)  The
+ITS names should be obvious, things like HSTLCS >, HSTAI >, etc, and
+the conventions on XX are a direct mapping of the MC filenames.
+Doesn't really matter so long as responsible people tell whoever is
+generating HOSTS3 files where everything is and what it is called.
+
+The NET entries for MIT will be moved into a seperate file of their
+own, MC: SYSHST; HSTNET > or something like that.  To avoid name
+collisions, The HOSTS3 compilation will also be using a hacked up
+version of HSTNIC > with all the MIT entries removed (this is one of
+the reasons why the MIT NET entries go in a seperate file).  Again,
+all this will be done by some hairy XFILE or .CTL file and shouldn't
+need human attention unless it breaks.  But it will be somewhat
+compute bound.....
+
+That should be basicly it.  Updates will of course be posted to
+INFO-HOSTS@MC.

doc/syshst/-what-.-file-

@@ -0,0 +1,203 @@
+XX:<HOSTS>-WHAT-.-FILE-.1,  5-Sep-86 15:21:21, Edit by SRA
+
+This file attempts to document the important files related to MIT host
+tables, where the files come from, where they go, etcetera.  No
+warranty implied....
+
+
+Which tables:
+
+	The "MIT host tables" include information for the MIT
+	chaosnet and Internet networks 18.0.0.0 and 128.52.0.0.  There
+	are also a few overflows from the Arpanet (10.0.0.0) and in
+	past days Symbolics and CISL have also been part of the tables
+	for various reasons.  The "MIT host tables" are the
+	authoritative source of information for the MIT Chaosnet.
+	They also contain authoritative information for much of the
+	relevant Internet data, but here the organization is not as
+	clean; you should also check with the MIT Telecommunications
+	Office, who run the MIT.EDU domain and the campus network.
+	Contact Jeff Schiller <jis@bitsy.mit.edu> for details.
+
+Directories:
+
+	Host tables primarily live in the SYSHST directory on AI
+	(AI:SYSHST;) and in the HOSTS directory on XX (XX:<HOSTS>).
+	Other machines will presumably have local copies of these.
+	Some of the files also live in XX:<SYSTEM>; these are ones
+	that are used directly by XX's system programs.  Similarly,
+	the binary HOSTS3 file lives in the SYSBIN directory on all
+	ITS machines.
+
+
+Programs:
+
+Most relevant programs are in XX:<HOSTS>.  I used various languages as
+was convient (TECO, Twenex CTL, Twenex PCL, Twenex CMD, C, MIDAS).
+You are welcome to read this stuff but probably don't want to touch it
+unless you are a Real Hero.
+
+
+Mailing lists:
+
+The primary list for host table related issues is INFO-HOSTS@AI (aka
+AI.AI.MIT.EDU).  There are a couple of spinoff lists.
+INFO-HOSTS-UPDATE@AI is a list for people who want to receive SRCCOMs
+of changes to HSTMIT from the nightly XX batch job.
+INFO-HOSTS-REQUEST is the usual list addition/removal address.
+NAMECALLERS@AI is a list to discuss the MIT implementation of the
+DARPA Domain system.  You're welcome to listen in but don't speak
+unless you know what you are talking about.
+
+Source files:
+
+XX:<HOSTS>HOSTS.NIC:
+	This is a copy of the Internet host table maintained by the
+	Network Information Center under DARPA contract.  It is the
+	closest thing available to a complete listing of the Internet,
+	but is somewhat deficient (due to the unmanageable size of the
+	table).  The version number of this file tracks the version
+	number of [SRI-NIC]NETINFO:HOSTS.TXT.  MIT machines desiring a
+	copy of the unadulterated NIC table should copy the one from
+	XX rather than getting it directly from the NIC; this was an
+	official request from the NIC, to help cut down on the load on
+	SRI-NIC due to FTP servers.
+
+	To update this file, send mail to HOSTMASTER@NIC.
+
+AI:SYSHST;HSTNET >
+	This file contains about ten lines of machine readable data
+	(network names and numbers for MIT nets).  This part is seldom
+	changed.  Most of the file is a comment (ignored by the table
+	compilers) listing the current subnet assignments for the MIT
+	networks.  This is the authoritative listing.  No changes
+	should be made to subnet numbers without checking this file to
+	see if there is a conflict.  There is a small self appointed
+	militia who police this file for reasonableness; talk to the
+	people on INFO-HOSTS@AI if you need details.
+
+AI:SYSHST;HSTLCS >
+	This is the authoritative host table for LCS.  It is dumped
+	from the LCS LispM namespace every time a "significant" change
+	is made to the namespace.  Editing the file is a waste of
+	time, edit the namespace instead.  Format is a little odd to
+	support some additional functionality for the LCS.MIT.EDU
+	domain.  A copy of this file lives in XX:<HOSTS>HSTLCS.TXT.
+	This file is also the source for the LCS.MIT.EDU domain.
+
+AI:SYSHST;HSTAI >
+	This is the authoritative host table for the AI lab.  It is
+	dumped from the AI LispM namespace.  Again, edit the namespace
+	instead of the file.  File format doesn't have the weird LCS
+	kludges.  A copy of this file lives in XX:<HOSTS>HSTAI.TXT.
+	This file contains the same data as the AI.MIT.EDU domain.
+
+AI:SYSHST;HSTEE >
+	Table for the EECS "domain" (still part of MIT.EDU).  Hand
+	edited, but usually pretty up to date.  Is the authoritative
+	source for EECS Chaosnet machines.  Internet addresses here
+	have to be cleared with Telecommunications, but changes should
+	be reflected in this file so that machines which don't grok
+	domains yet can keep up with things.  Copy in
+	XX:<HOSTS>HSTEE.TXT.
+
+AI:SYSHST;HSTATH >
+	Table for the Project Athena "domain" (still part of MIT.EDU).
+	Hand edited, may or may not be up to date at a given moment.
+	Authoritative source for Athena Chaos machines.  Copy lives in
+	XX:<HOSTS>HSTATH.TXT.
+
+AI:SYSHST;HSTG >
+	Table for machines not part of LCS, AI, Athena, or EECS.
+	Hand edited, may or may not be up to date.  Authoritative for
+	Chaosnet.  Copy in XX:<HOSTS>HSTG.TXT.
+
+AI:SYSHST;HSTXXX >
+	This is a file of additions and kludges that need to be added
+	to the NIC host table (some machines at Stanford and
+	Symbolics, other things as needed).  You probably shouldn't
+	edit this.
+
+Generated files (produced by XX's nightly host table job):
+
+XX:<HOSTS>HSTMIT.TXT
+	Merge of HSTLCS, HSTAI, HSTATH, HSTEE, and HSTG.  This has all
+	the special kludges fixed up, names expanded, etcetera.  This
+	is the closest thing there is to a single table listing all
+	MIT machines.  Copy in AI:SYSHST;HSTMIT >.
+
+XX:<HOSTS>HSTNIC.TXT
+	This is derived from the NIC host table.  A few known bad
+	names and/or addresses are removed, any nicknames that
+	conflict with MIT machines are removed.  Any machines that
+	look "insignificant" are removed (workstations at sites other
+	than MIT, etcetera).  The contents of this file are subject to
+	change without notice as becomes necessary to feed this file
+	into some of the later stages of the compilation process.
+	HSTXXX is prepended to the NIC data in this file.  No data for
+	MIT machines appears in this file; it is implicitly assumed
+	that the data in the MIT tables superceedes whatever the NIC
+	has on file.
+
+XX:<SYSTEM>HOSTS.TXT
+	This is HSTMIT followed by HSTNIC, but without the Chaosnet
+	data.  It is as close to a "complete" listing of the DARPA
+	Internet as can be made with the current setup.  XX uses this
+	file for its old Internet host table software (XX also uses
+	the new Domain system, but that's a different issue).
+	Machines that want a copy of the NIC table with the MIT data
+	fixed should use this file.
+
+XX:<HOSTS>HOSTS3.TXT
+	This is a file suitable for feeding to the HOSTS3 binary table
+	compiler.  It contains all the essential data from HSTMIT
+	followed by HSTNIC.  Some amount of pruning is done to this
+	file to fit the address space constraints of the HOSTS3
+	compiler (at present, all the services entries are removed).
+	Contents subject to change without notice.
+
+XX:<SYSTEM>HOSTS3.BIN
+	This is the compiled binary HOSTS3 table.  Many programs on
+	ITS and Twenex use this.  Copies can be found in the <SYSTEM>
+	directory on all MIT Twenex machines and in the SYSBIN;
+	directory on all ITS machines.  Version numbers of the ITS
+	copies track that of the XX:<SYSTEM> copy.
+
+XX:<HOSTS>HOSTS2.TXT
+	An old format of host table (MIDAS hackers should note the
+	syntax!).  Somewhat useless because it can only handle IP
+	addresses with the third octet zeroed.  Used to produce...
+
+XX:<SYSTEM>HOSTS2.BIN
+	The compiled binary HOSTS2 format file.  Maintained for upward
+	compatibility.  No new programs should use this.
+
+XX:<HOSTS>HOST3C.TXT
+	A listing of just the Chaosnet machines from HSTMIT.  Name is
+	somewhat confusing, since the file format is actually HOSTS2.
+	I think the unix binary table compiler reads this file.  Is
+	called HOST3C because it is used to produce...
+
+XX:<SYSTEM>HOST3C.BIN
+	A HOSTS3 binary format listing of the MIT Chaosnet.
+
+XX:<HOSTS>MITGWS.TXT
+	A listing of (IP) gateways and (Chaos) bridges, produced from
+	HSTMIT.  No warranty, but very handy for tracking down
+	network problems, since it is usually more up to date than any
+	of the gateway maps available.  Copy in AI:SYSHST;MITGWS >.
+
+XX:<HOSTS>CHAOSHOSTS.CHAOS-ALL
+	A file in the format used by unix sendmail, usually installed
+	as /etc/chaoshosts.  This file contains a listing of all
+	chaosnet hosts at MIT (derived from HSTMIT).
+
+XX:<HOSTS>CHAOSHOSTS.CHAOS-ONLY
+	Same as CHAOSHOSTS.CHAOS-ALL but only lists hosts that are on
+	the chaosnet but aren't on the Internet.  Depending on whether
+	your vax is or isn't on the net you will want one or the other
+	of these files so that sendmail can route mail as efficiently
+	as possible.
+
+There are some other files associated with the LCS.MIT.EDU domain, but
+that's a whole separate topic and not relevant here.

doc/syshst/hosts3.readme

@@ -0,0 +1,10 @@
+If you are looking for the source code to the HOSTS3 program because you
+read a comment in SYSENG;NETWRK, be informed that the latest version
+because the latest version is the one that runs on XX at 4am via batch
+job and it may or may not compile correctly on ITS because the page
+granularity is finer on 20x and at one point I needed every word of
+address space I could get out of the program.  See the file named (from
+ITS EMACS) XXSRC: SYS.SYSTEM; HOSTS MID if you care.  I don't think
+there are any changes other than address space twiddles.
+
+--sra 12 Nov 86