diff --git a/build/build.tcl b/build/build.tcl index 31dde894..1dff265f 100644 --- a/build/build.tcl +++ b/build/build.tcl @@ -154,6 +154,7 @@ if {$env(BASICS)!="yes"} { source $build/scheme.tcl source $build/muddle.tcl source $build/sail.tcl + source $build/typeset.tcl } bootable_tapes diff --git a/build/typeset.tcl b/build/typeset.tcl new file mode 100644 index 00000000..af3a3651 --- /dev/null +++ b/build/typeset.tcl @@ -0,0 +1,5 @@ +log_progress "ENTERING BUILD SCRIPT: TYPESET" + +respond "*" ":bolio;bolio moon;amber\r" +respond "NIL" "(quit)" +expect ":KILL" diff --git a/doc/moon/amber.66 b/doc/moon/amber.66 new file mode 100644 index 00000000..7233be26 --- /dev/null +++ b/doc/moon/amber.66 @@ -0,0 +1,1572 @@ +.c -*-TEXT-*- + +.c Note that this file will eventually be conditionalized to turn into +.c two versions, one a publishable paper and the other the complete +.c user and implementer reference manual for the network. + +[Notes: + Insert references + Fix bracketed commentary + Check all quoted numbers + Make sure all initialisms are decoded upon first occurence. + Always spell it Chaosnet. + Work supported by.] + +.chapter The Chaosnet + +Chaosnet is a 2local network*, that is, a system for communication +among a group of computers located within about 1000 meters of each other. +The name Chaosnet refers to the lack of any centralized control element +in this network. + +Chaosnet was originally developed by the Artificial Intelligence Laboratory +of the Massachusetts Institute of Technology as the internal communications +medium of the Lisp machine system [ref]. It has since come to be used to +link a variety of machines around the Institute. + +The Lisp machine system is a multi-processor in which each active user +is assigned a "personal" computer consisting of a medium-scale +processor, a suitable amount of memory, and a swapping disk. Files are +stored in a central file-system accessed through the Chaosnet. This +shared file-system retains the traditional advantages of a time-sharing +system, namely inter-user communication, shared programs, and +centralized maintenance. This system is intended to execute Lisp +programs several million words in size efficiently and with rapid +interactive response. Because the Chaosnet is taking the place of the +file disk in a conventional system, it must be fast (both in response +and in throughput), it must be reliable (this is the reason why there +is no centralized control), and it must allow connection of several +dozen machines. However, it does not need to operate over long +distances. + +[More about system goals?] + +Chaosnet consists of two parts--the hardware and the software--which, +while logically separable, were designed for each other. The hardware +provides a structure very similar to the PARC Ethernet [ref.]. Network +nodes contend for access to a single cable, or ether, over which they +may transmit packets addressed to other network nodes. The software +defines higher-level protocols in terms of packets. These protocols +can be (and are) used with media other than the Chaosnet cable. +The software contains ideas borrowed from Ethernet [ref.], TCP [ref.], +and Arpanet [ref.], with some original ideas and modifications. + +Currently (March 1979) the Chaosnet at MIT is connected to four Lisp machines, +three pdp10 timesharing systems--two running MIT's ITS operating system +and one running DEC's TOPS-20 operating system, and a terminal concentrator +with three conventional terminals, eight raster-scan displays, and a printer/plotter. + +.chapter Hardware Protocol + +.section The Ether + + The transmission medium of the Chaosnet is called the +2ether*. Physically it is a coaxial cable with 75-ohm terminations +at the ends. [Get cable type-name] At each network node a 2cable +transceiver* is attached to the cable. A 10-meter flat cable connects +the transceiver to an 2interface* which is attached to the computer's I/O bus. + + One network node at a time may seize the ether and transmit a +packet, which arrives at all other nodes; each node decides in hardware +whether to ignore the packet or to receive it. A single ether must be +a linear cable; it may not contain branches nor stubs, and the ends may +not be joined into a circle. The maximum length of a single ether +cable is about 1 kilometer. This is determined by dispersion and by DC +attenuation. The maximum number of nodes on a single ether is probably a few dozen. +This is determined by degradation of the electrical properties of the cable +by the connectors. +The protocol provides for multiple ethers (or other +media), joined together by nodes called 2bridges* which relay +packets from one ether to another. + + The form of information on the ether, the transceiver and +interface hardware, and the protocols which control who may seize the +ether are described in the following sections. + +.section Packets + + The basic unit of transmission is called a packet. This is a +sequence of up to 4032 data bits, plus 48 bits of 2header* information +used by the hardware. Packets' bits are normally grouped into 16-bit +words. The division of the transmitted bit stream into packets provides a +conveniently-sized unit for resource allocation and error control. + + The hardware header consists of three 16-bit words, called +2destination*, 2source*, and 2check*. The source identifies the node +which transmitted this packet onto this ether. This is not necessarily the +original source of the message, since it may have originated on a +different ether. The destination identifies the node which is intended +to receive this packet from this ether. This is not necessarily the final +destination of the message; it may be a bridge which should relay the +packet to another ether, eventually reaching the final destination. +The check word is a cyclic redundancy checksum, generated and checked +by hardware, which detects errors in transmission through the ether, +entirely-spurious packets created by noise on the cable, and memory +errors in the transmitting and receiving packet buffers. + + The software protocol is also based on packets, taking 128 of the data bits +as a software header. This is described later. + +.section The Transceiver + + Everyone who connects to the ether does so through a transceiver, +which is a small box which mounts directly on the cable via a UHF connector and a T-joint. +The transceiver contains the analog portion of the interface logic, provides +optical isolation between the ether cable and the computer, and contains some +protective circuitry designed to prevent a malfunctioning program or interface +from continuously jamming the ether. + + The transceiver receives a differential digital signal from the computer interface +and impresses it onto the cable as a level of about 8 [??] volts for a 1, +and ground (open circuit) for a 0, through a very fast VMOS power FET. +When the cable is idle it is held at ground by the terminations. This +simple-minded unipolar scheme is adequate for the medium cable lengths and +transmission speeds we are using. The transceiver monitors the cable by +comparing it against a reference voltage, and returns a differential signal signal +to the interface. In addition, it detects interference (another transceiver +transmitting at the same time as this one) and informs the interface. + + The transceiver includes indicators (light-emitting diodes) for +power OK, transmitted data, received data, and interface attempting to +jam the ether. A test button simulates an input of continuous 1's from +the interface, which should light all the lights (dimly) if +the transceiver is working. These indicators and the test button +are useful for rapidly tracking down network problems. +[Ref?] + +.section The Interface + + The interface is a wire-wrap board containing about 120 TTL +logic chips, which plugs into the I/O bus of a computer and connects it +to the ether (through a transceiver.) The interface implements the hardware +protocols described in the next section, buffers incoming and outgoing +packets, generates and checks checksums, and interrupts the host computer +when a packet is to be read out of the receive packet buffer or stored +into the transmit packet buffer. + + Our current interface plugs into a pdp-11 Unibus [ref.] which +allows it to be attached to pdp-11's, Lisp machines, VAX's, and the +peripheral processors of most pdp-10's. The remaining 60 DIP positions +of the Chaosnet interface board are filled out with several other small +I/O devices, for the benefit of Lisp machines. Programming +documentation for this interface appears later in this paper. + +A Chaosnet interface for the Interdata I/O bus has been designed at the +MIT Architecture Machine Group but is not yet constructed. + +.section Hardware Protocols + + The purpose of these protocols is to deliver packets intact +from one interface to another interface on the same ether, with fairly +high probability of success, and to guarantee to give an error indication or lose +the packet entirely if it is not delivered intact. An additional purpose +is to provide high performance and not to bog down when subjected to +a heavy load. + + Bits are represented on the ether using a technique which is +[called something or other like Upright Bi Phase NRZI or something like +that.] Each bit cell, which is approximately 125 nanoseconds long, +begins with a transition in state, from high to low or from low to high. +This transition marks the beginning of a bit cell and provides self-clocking. +3/4 of the way through the bit cell, the state of the cable is sampled; +high represents a 1 and low represents a 0. If the bit being represented +is the same as the previous bit, there will be one transition at the beginning +of the bit cell and a second in the middle of the bit cell. If the bit +being represented is the opposite of the previous bit, there will be no +transition in the middle of the bit cell since the clock transition will +have set the cable to the desired state. The frequency of the signal on +the cable varies between 1/2 the bit rate and the full bit rate. +The information bit-rate is 8 million bits per second. + + The self-clocking feature allows for slight variations in transmission +and cable propagation speed. However, since the 3/4 of a bit cell delay +is a fixed delay, only small variations in speed can be tolerated. A crystal +clock is used as the source of the transmit timing in the interface. + + Since there is always at least one state-transition per bit cell, +the states where the ether remains high or low for an appreciable time +are available for non-data uses. If the ether remains low for more than +about 1/2 microsecond, it is considered to be not-busy. This condition marks +the end of a packet and allows someone else to transmit. Note that if no +transceivers are active, the terminations will hold the ether low. + + If the ether remains high for about 1 microsecond, this is an +"abort signal". Abort signals are used for two purposes. If the +transceiver detects a collision (two nodes trying to transmit at the +same time), each transmitting interface ceases to transmit and sends an +abort signal, which tells all receivers to ignore the aborted packet +and ensures that the other transmitter also aborts. Thus when a +collision occurs, the ether is cleared as soon as possible to help +prevent long tie-ups under conditions of heavy load. The other use for +the abort signal is in hardware flow-control. When a receiving +interface determines that an incoming packet is addressed to it, but +its receive buffer already contains a packet, it sends an abort signal +which causes the transmitter to stop. This serves the dual purpose of +immediately informing the transmitter that its message did not get +through, and preventing the ether from being tied up while the full +length of a packet is transmitted which the receiver cannot receive. + + Packets are transmitted over the ether in reverse bit-order, for +hardware convenience. The three header words, which to the software appear +to be at the end of the packet, are transmitted first, in the order +check, source, destination. The data words, in reverse order, follow. +Words are transmitted least-significant bit first. At the end of the +packet, an extra zero bit is appended to bring the ether to the low +state so that no clock-transition will be generated when it goes idle. +This bit is stripped off by the interface and never seen by software. + + The check word is used for error detection, as described above. +The source word is made available to the software, which ignores it in +most cases, and also serves to synchronize the clocks in the +collision-avoidance mechanism which is described below. The +destination word is compared by each receiver against its own address. +If they match, or if the destination is zero, or if the software +selects the "match any destination" mode, the packet is placed in the +receive packet buffer and the host computer is interrupted. The zero +destination feature is used for "broadcast" messages. Note that a +receiver whose packet buffer is full will only generate an abort signal +if the packet was specifically addressed to it. + +.section Ether Contention + + Chaosnet has no centralized control element; when a network node +has a message to transmit, its interface seizes the ether and transmits +a packet. The time when it seizes the ether is determined only by state +inside that particular interface and by the local state of the cable at +the point where that interface is attached. + + If two interfaces should decide to seize the ether and transmit +at the same time, their transmissions will interfere and no useful +information will be transmitted. This is called a 2collision*. +Collisions are the principal limitation on the bandwidth of a +heavily-loaded ether-type network, and should be avoided. (However, +neither PARC's network nor MIT's network has yet been operated with a +heavy enough load to make collisions really significant.) + + Chaosnet uses a novel collision avoidance technique. First of +all, an interface will never initiate transmission unless the ether is +seen to be not busy, i.e. it has been in the low state for some time. +This ensures that collisions can only occur near the beginning of a +packet. Once transmission of a packet has gotten well started, the +ether is effectively "seized" (all interfaces realize that it is busy) +and transmission will continue successfully through to the end of the +packet. The amount of ether transmission time wasted by a collided +packet is therefore limited to the round-trip cable propagation delay. + + Secondly, the hardware uses a time-division technique to attempt +to prevent two interfaces from initiating transmission at the same time. +This technique should prevent essentially all collisions while imposing +only a modest delay in the initiation of transmission. It is designed +so that it works better as the load on the ether increases; the wasted time +between packets and the relative rate of collisions both decrease. + + The basic idea is that each interface is assigned a time-slot, +or 2turn*, according to its address. It may only initiate +transmission during its turn. The turns are spaced far enough apart +that if one interface initiates transmission, every other interface +will perceive that the ether is busy by the time its own turn arrives, +and will not initiate an interfering transmission. Each interface +contains a time-slot counter which counts while the ether is not busy, +keeping track of whose turn it is. Each packet synchronizes the +counters in all of the interfaces by setting them from the source +address of that packet; at the time the packet was transmitted, it must +have been the turn of the interface that transmitted it. + + Another way to think of this is to make an analogy with ring +networks [ref]. One can imagine a 2virtual token* which passes down the cable +until it gets to the end, then jumps to the beginning of the cable and +repeats. An interface may only initiate transmission at the instant the +token passes by it. When an interface transmits, the token stops moving +and remains at that interface until the end of the packet, whereupon it +continues down the cable, passing every other interface, giving them +each a chance to transmit before letting the first interface transmit a second packet. + + The token is not represented by any physical transmission on the +cable. That would constitute a form of centralized control, and would +lead to reliability problems if the token was lost or duplicated. +Instead, every interface contains a time-slot counter which keeps +track of where the token is thought to be. Every time a packet is +transmitted these counters are brought up-to-date. The token cannot be +lost because a counter by its nature eventually returns to all previous +states. It does not matter if the token is duplicated (that is, the +counters lose synchronization) occasionally, since this will only cause +collisions, which we know how to detect and deal with, and since the first successful +transmission will resynchronize all counters. The basic mechanism +of the ether network with contention and collisions is known to work, +and the collision-avoidance scheme is an added-on optimization which +improves performance without changing the basic mechanism. + + There is a finite propagation delay time between interfaces, +and this time is not small compared with the bit-rate of the Chaosnet. +This time consists of the delay in the cable, about 5 nanoseconds per meter, +and the delay through the two transceivers, about 220 nanoseconds. +This propagation delay means that the time-slot counters in two different +interfaces cannot be exactly synchronized, and that when interface A +initiates transmission interface B will not instantaneously see that the +ether is busy. Special relativity [ref!!?!] tells us that in fact the concept +"exactly synchronized" is meaningless. Since the two time-slot counters are +not in the same place, the only way we can compare them is to send a +message from one to the other, through the ether, containing the reading of +the counter. But this message takes non-zero time to get there, so the +counter-reading it contains is wrong by the time it is compared against the +other counter! We in fact do send messages containing counter readings; +the source address in a packet equals the reading of the time-slot counter in the +interface that sent it--at the time it was sent. + + What we are trying to do is to prevent collisions. This means +that if interface A starts transmitting a packet in its turn, then by +the time interface B thinks that its own turn has arrived, it must +perceive the ether as busy. We will assign addresses (and hence time +slots) and set the length of a time slot in such a way that this will +happen. Suppose the maximum delay through the ether between A and B is T. +This would be the delay for one of them sending a packet to the other; +the delay between A's receipt of a third party's packet and B's receipt +of that packet is less, especially if the third party is between A and +B on the cable. Then the maximum perceived difference between a clock at A +and a clock at B is 2T; if a message is sent from B to A synchronizing +the clocks, and then a message is sent from A to B containing A's clock +reading, at B this clock reading will be slow by 2T. + + When a packet transmitted by A arrives at B, B's clock may read as +much as 2T earlier or later than A's turn, depending on the transmission +direction of the last synchronizing message. In order to guarantee that B's +turn has not yet happened, the time between any of A's turns and any of B's +turns must always be at least 2T, twice the maximum propagation delay +through the ether between A and B. This is the important idea! We cause +this to be true by assigning addresses starting at one end of the cable; +each machine's address is the previous machine's address plus twice the +propagation delay between them, divided by the length of a turn. It is +easy to see that if this is done for all adjacent pairs, the condition will +automatically be true for non-adjacent pairs as well. When we get to the +end of the cable, we must assign a number of empty slots equal to twice the +propagation delay of the full length of cable, to provide +the necessary separation between the turns of the two machines at the ends +of the cable. + + The virtual token travels through the network at a substantially +slower speed than a real packet; in the fastest case, when nodes are very far +apart, it travels at just half the speed of a real packet. Since the +Chaosnet has the geometry of a line, as compared to the ring net's +circle, the virtual token is also slowed down by the need to return from +the end of the cable to the beginning. This slower speed of the token is +the price one pays for the increased robustness of a Chaosnet as +compared with a ring network. In a real system, we slow the token down +even more to provide a margin of safety. The speed of the network is not +significantly affected by the slow token, if it is being used primarily +for file transfer, since the transmission time for a maximum-length packet +is several times the round-trip time of the token. + + In spite of all this, sometimes collisions will occur anyway. If +the cable has been idle for a long time, the various clocks will have lost +synchronization. If a source address is corrupted by a transmission error, +any interface that sees that source address will set its clock to an +incorrect value. Sometimes a packet will collide with random noise rather +than another legitimate packet. In addition, the transmitter does not +distinguish receiver-busy aborts from real collisions. + + When a collision does occur, we recover from it (in software) +by retransmitting the packet again a couple of times, hoping that we +will be lucky enough not to have another collision, or that the +receiver will soon clear its packet buffer. This retransmission is +done by the software, not the hardware, since the hardware destroys the +packet in its packet buffer in the process of transmitting it. But if +collisions continue to occur, we give up and let somebody else have +the ether. The packet is lost. The software protocol will soon +realize that it has been lost and retransmit it. We assume that there +is enough randomness in the higher-level software that the two +computers which originally collided will not collide again on the +retransmission by deciding to retransmit at precisely the same instant. + + +.chapter Software Protocol--World View + +The purpose of the basic software protocol of the Chaos network +is to allow high-speed communication among processes on different +machines, with no undetected transmission errors. The speed for +file transfers was to be comparable with an inexpensive magnetic tape +drive (30000 characters per second, or about 10 times the speed of the +Arpanet). We actually get about double this in some cases. +To achieve this speed it was important to design out bottlenecks such +as are found in the Arpanet, for instance the control-link which is +shared between multiple connections and the need to acknowledge each +message before the next message can be sent. The protocol must be +simple, for the sake of reliability and to allow its use by modest +computer systems. A full Chaos Network Control Program is just about +half the size of an Arpanet NCP on the same machine, and the protocol +allows low-performance implementations to omit some features. + +.section Connections + +The principal service provided by Chaosnet is a 2connection* between +two user processes. This is a full-duplex reliable packet-transmission +channel. The network undertakes never to garble, lose, duplicate, or +resequence the packets; in the event of a serious error it may break +the connection off entirely. User programs may either deal in terms of +packets, or ignore packet boundaries and treat the connection as two +streams of 8-bit or 16-bit bytes. + +On top of the connection facility "user" programs build other +facilities, such as file access, interactive terminal connections, and +data in other byte sizes, such as 36 bits. The meaning of the packets +or bytes transmitted through a connection is defined by the particular +higher-level protocol in use. + +In addition to reliable communication, the protocol provides flow +control, includes a way by which prospective communicants may get in +touch with each other (called 2contacting* or 2rendezvous*), and +provides various network maintenance and housekeeping facilities. + +.section Contact Names + +When first establishing a connection, it is necessary for the two +communicating processes to contact each other. In addition, in the +usual user/server situation, the server process does not exist beforehand +and needs to be created and made to execute the appropriate program. + +We chose to implement contacting in an asymmetric way. (Once the +connection has been established everything is completely symmetric.) +One process is designated the 2user*, and the other is designated the +2server*. The server has some 2contact name* to which it +2listens*. The user process requests its local operating system to +connect it to the server, specifying the network node and contact name +of the server. The local operating system sends a message (a 2Request +for Connection*) to the remote operating system, which examines the +contact name and creates a connection to a listening process, creates +and connects to a new server process, or rejects the request. + +Automatically discovering to which host to connect in order to obtain a +particular service is a subject for higher-level protocols and for +further research. It is not dealt with by Chaosnet. + +Once a connection has been established, there is no more need for the +contact name and it is discarded. Indeed, often the contact name +is simply the name of a service (such as "7TELNET*") and several +users should be able to have simultaneous connections to separate +instances of that service, so contact names must be reusable. + +In the case where two existing processes that already know about each +other want to establish a connection, we arbitrarily designate one as +the listener (server) and the other as the requester (user). The +listener somehow generates a "unique" contact name, somehow +communicates it to the requester, and listens for it. The requester +requests to connect to that contact name and the connection is +established. In the most common case of establishing a second +connection between two processes which are already connected, the index +number (see below) of the first connection can serve as a unique +contact name. + +Contact names are restricted to strings of upper-case letters, numbers, +and ascii punctuation. The maximum length of a contact name is limited +only by the packet size, although the ITS file-system limits names of +automatically-started servers to six characters. + +.section Addresses and Indices + +Each node (or host) on the network is identified by an 2address*, +which is a 16-bit number. These addresses are used in the routing of +packets. There is a table (the system hosts table, 7SYSBIN;HOSTS2*, in the case +of ITS) which relates symbolic host names to numeric host addresses. + +An address consists of two fields. The most-significant 8 bits +identify a 2subnet*, and the least-significant 8 bits identify a host +within that subnet. Both fields must be non-zero. A subnet +corresponds to a single transmission path. Some subnets are physical +Chaosnet cables (2ethers*), while others are other media, for +instance an interface between a pdp10 and a pdp11. + +When a host is connected to an ether, the host's hardware address on that +ether is the same as its software address, including the subnet field. + +A connection is specified by naming its two ends. Such a name consists +of a 16-bit host address and a 16-bit connection index, which is assigned +by that host. The only requirements placed by the protocol on indices +are that they be non-zero and that they be unique within a particular +host; that is, a host may not assign the same index number to two different +connections unless enough time has elapsed between the closing of the first +connection and the opening of the second connection that confusion between +the two is unlikely. + +Typically the least-significant 2n* bits of an index are used as a +subscript into the operating system's tables, and the most-significant +16-2n* bits are incremented each time a table slot is reused, to +provide uniqueness. The number of uniquizing bits must be sufficiently +large, compared to the rate at which connection-table slots are reused, +that if two connections have the same index, a packet from the old +connection cannot sit around in the network (e.g. in buffers inside +bridges) long enough to be seen as belonging to the new connection. + +.section Packet Numbers + +There are two kinds of packets, controlled and uncontrolled. Controlled +packets are subject to error-control and flow-control protocols, described +below, which guarantee that each controlled packet is delivered to +its destination exactly once, that the controlled packets belonging +to a single connection are delivered in the same order they were sent, +and that a slow receiver is not overwhelmed with packets from a fast sender. +Uncontrolled packets are simply transmitted; they will usually but not +always arrive at their destination exactly once. The protocol for using +them must take this into account. + +Each controlled packet is identified by a 16-bit 2packet number*. +Successive packets are identified by sequential numbers, with wrap-around +from all 1's to all 0's. When a connection is first opened, each end +numbers its first controlled packet (RFC or OPN) however it likes, and +that sets the numbering for all following packets. + +Packet numbers should be compared modulo 65536 (2 to the 16th), to ensure correct +handling of wrap-around cases. +On a pdp11, use the instructions +.lisp +CMP A,B +BMI A_is_less +.end_lisp +Do not use the 7BLT* or 7BLO* instruction. +On a pdp10, use the instructions +.lisp +SUB A,B +TRNE A,100000 + JRST A_is_less +.end_lisp +Do not use the 7CAMGE* instruction. +On a Lisp machine, use the code +.lisp +(AND (BIT-TEST 100000 (- A B)) + ) +.end_lisp +Do not use the 7LESSP* (or 7<*) function. + +.section Packets + +A packet consists of a header, which is 8 16-bit words, and zero or +more 8-bit or 16-bit bytes of accompanying data. In addition there are +three words put on by the hardware, described earlier in this paper. + +The following are the 8 header words: +.table 2 0 1000 0 120 +.item Operation +The most-significant 8 bits of this word are the 2Opcode* of the packet, +a number which tells what the packet means. The 128 opcodes with high-order +bit 0 are for the use of the network itself. The 128 opcodes with high-order +bit 1 are for use by users. The various opcodes are described in the Packet Types section. + +The least-significant 8 bits of this word are reserved for future use, +and must be zero. +.item Count +The most-significant 4 bits of this word are the forwarding count, which tells +how many times this packet has been forwarded by bridges. Its use is explained +in the Routing section. + +The least-significant 12 bits of this word are the data byte count, +which tells the number of 8-bit bytes of data in the packet. The +minimum value is 0 and the maximum value is 488. Note that the count +is in 8-bit bytes even if the data are regarded as 16-bit bytes. +.item Destination Address +This word contains the network address of the destination host +to which this packet should be sent. +.item Destination Index +This word contains the connection index at the destination host +of the connection to which this packet belongs, or 0 if this packet +does not belong to any connection. +.item Source Address +This word contains the network address of the source host which +originated this packet. +.item Source Index +This word contains the connection index at the source host +of the connection to which this packet belongs, or 0 if this +packet does not belong to any connection. +.item Packet Number +If this is a controlled packet, this word contains its identifying number. +.item Acknowledgement +The use of this word is described in the Flow Control section. +.end_table + +.section Data Formats + +Data transmitted through Chaosnet generally follow Lisp machine +standards. The Lisp machine character set [ref.] is used, except when +a higher-level protocol such as Telnet or Supdup explicitly specifies a +different character set. Bits and bytes are numbered from right to +left, or least-significant to most-significant. The first 8-bit byte +in a 16-bit word is the one in the arithmetically least-significant +position. + +Because this conflicts with the native byte-ordering in pdp10s, and because it +is quite expensive to rearrange the bytes using the pdp10 instruction set, pdp11s +which act as front-ends for pdp10s must reformat packets passing through them. +This requires that the network protocols explicitly specify which portions of +each type of packet are 8-bit bytes and which are 16-bit bytes. In general the +header is 16-bit bytes and the data field is 8-bit bytes, but certain packet +types (OPN, STS, RUT, and opcodes 300 through 377) have 16-bit bytes in the +data field. + +.section Routing + +2Routing* consists of deciding how to deliver a packet to the network +node specified by the destination address field of the packet. +In general this may be a multi-step process involving transmission +through several subnets, since there may not be a direct hardware +connection between the source and the destination. Note that the +routing decision is made separately for each packet, with no reference +to the concept of connections. + +Any host that is connected to more than one subnet acts as a 2bridge* +and 2forwards* packets from one subnet to another when necessary. +There could also be hardware bridges which are not hosts, although we have +not yet designed any such device. The difference between a 2bridge* +and a 2gateway*, in our terminology, is that a bridge forwards packets +from one sub-Chaosnet to another, without modifying the packets or understanding +them other than to look at the destination address and increment the forwarding count, +and does not +deal with connections nor with flow control, while a gateway interconnects +two networks with differing protocols and must understand and translate the +information passing through it. + +To prevent forwarding loops, each packet contains a forwarding-count field. +Each bridge that forwards the packet increments this count; if the count reaches +its maximum value the packet is discarded. The error-control protocol will +recover discarded packets, or decide that no viable connection can be established +between the two hosts. + +The implementation of routing in an operating system is as follows, given a packet +to be routed, which may have come in from the network or may have been originated +by the local host. First, increment the forwarding count and discard the packet +if it has been forwarded too many times. Next check the packet's destination address. +If it is this host, receive the packet. If it is some other host on a subnet +to which this host is directly connected, transmit the packet on that subnet; +the destination host should receive it. If the destination is a host on a subnet +of which this host has no knowledge, look up the subnet in the host's 2routing table* +to find the best bridge to that subnet, and transmit the packet to that bridge. + +Each host has a routing table, indexed by subnet number, which tells +how to get packets to hosts on that subnet. +Each entry contains: (exact details may vary depending on implementation) +.table 2 +.item Type +The type of connection between the host and this subnet. This can +be one of 2Direct*, 2Bridge*, or 2Fixed Bridge*. 2Direct* means a physical +connection such as a Chaosnet interface. 2Bridge* means +an indirect connection, via a packet-forwarding bridge. Which bridge +is best to use is to be discovered by this routing mechanism. +2Fixed Bridge* is the same except that the automatic mechanism +is not to change which bridge is used. This is useful to set up +explicit routing for purposes such as network debugging. +.item Address +Identifies the connection to this subnet in a way which depends on the type. +For a direct connection, this identifies the piece of hardware which +implements the connection. (It might be a Unibus address.) +For a bridge or a fixed bridge, this is the network address of the bridge. +.item Cost +A measure of the cost of sending a packet through this route. +Costs are used to select the best route from among alternatives in a way +described below. For a direct connection, the cost is 1 for a direct +interface between two computers (e.g. between a pdp10 and its front-end pdp11), +2 for a Chaosnet ether cable, 5 for a slow medium such as an asynchronous line, etc. +For a bridge or a fixed bridge, the cost is specified by the bridge in a RUT packet +(described below). +.end_table + +The routing table is initialized with the number of a more or +less arbitrary existent host and a high cost, for each subnet +to which the host is not directly connected. Until the correct +bridge is discovered (which normally happens within a minute +of coming up), packets for that subnet will be bounced off of +that arbitrary host, which probably knows the right bridge to forward them to. + +The Cost for subnets accessed via bridges is increased by 1 every 4 +seconds, thus typically doubling after 15 seconds. When the cost +reaches a "high" value, it sticks there, preventing problems with +arithmetic overflow. The purpose of the increasing cost is to discount +the value of old information. The cost for subnets accessed via direct +connections and fixed bridges does not increase. + +Every 15 seconds, a bridge advertises its presence by broadcasting a +routing (RUT) packet on each subnet to which it is directly connected. +Each host on that subnet receives the RUT packet and uses it to update +its routing table. If the host's routing table says to access a certain +subnet via bridges, and the RUT packet says that this is the best bridge +to that subnet, the routing table is updated to say that this bridge should +be used. + +The source address of a RUT packet must be the hardware address of the +bridge on the particular subnet on which the packet is broadcast. The +destination address of a RUT packet must be zero; RUT packets are +not forwarded onto other subnets. The byte count of a RUT packet is a +multiple of 4 and the packet contains up to 122 pairs of 16-bit words: +.table 1 +.item word 1 +The subnet number of a subnet which this bridge can get to, +directly or indirectly, right-adjusted. +.item word 2 +The cost of sending to that subnet via this bridge. This is the +current cost from the bridge's routing table, plus the cost +for the subnet on which the routing packet is being broadcast. +Adding the subnet cost eliminates loops, and prefers one-hop +paths over two-hop paths. +.end_table + +When a host receives a RUT packet, it processes each 2-word entry by +comparing the cost for that subnet against its current cost; if it is less or equal +the cost and the address of the bridge are entered into the routing table, +provided that that subnet's routing table entry is not of the Direct +or Fixed Bridge type. + +When there are multiple equivalent bridges, the traffic is spread among them only by +virtue of their RUT packets being sent at different times, so that sometimes one +bridge has the lower cost, and sometimes the other. If this isn't adequate, +hosts could have hairier routing tables which remember more than one possible +route and use them according to their relative costs, but probably +this will not be necessary. +[***] + +The design of this routing scheme is predicated on the assumption that +the network geometry is simple, there are few multiple paths, +and the length of a path is quite short. This makes more sophisticated +schemes unnecessary. + +An important feature of this routing scheme is that the size of the +table is proportional to the number of subnets, not to the number of hosts. + +In the case of a pdp10 which accesses the net through a front-end pdp11, we define +the interface between the two computers to be a subnet, and regard the pdp11 +as a bridge which forwards packets between the network and the pdp10. +This gives the pdp10 and the pdp11 separate addresses so that we can +choose to talk to either one, even though they are part of the same +computer system. This is occasionally useful for maintenance purposes. + +In the case of a host which is connected to more than one subnet, +it is undesirable for the host to have more than one address, since this +would complicate user programs which use addresses. Instead, one of +the host's network connections is designated as primary, and that address +is used as the host's single address. The other connections are regarded +as bridges which can forward to that host. Often, we simplify the +routing by inventing a new subnet which contains only that host and +has no physical realization. The host's address is an address on that +fake subnet. All of the host's network connections are regarded as bridges +which know how to forward packets to that subnet. + +.section Flow and Error Control + +The Network Control Programs (NCPs) conspire to ensure that data +packets are sent from user to user with no duplications, omissions, or +changes of order. Secondarily, the NCPs attempt to achieve a maximum +rate of flow of data, and a minimum of overhead and retransmission. + +The fundamental basis of flow-control and error-control in Chaosnet is +2retransmission*. Packets which are damaged in transmission, which won't +fit in buffers, which are duplicated or out-of-sequence, or which otherwise +are embarrassing are simply discarded. Packets are periodically retransmitted +until an indication that they have been successfully received is returned. +This retransmission is end-to-end; any intermediate bridges do not participate +in flow-control and error-control, and hence are free to discard any packets +they wish. + +There are actually two kinds of packets, 2controlled* and 2uncontrolled*. +Controlled packets are retransmitted and delivered reliably; most +packets, including all packets used by the user, are of this type. +Uncontrolled packets are not retransmitted; these are used for certain +lower-level functions of the protocol. The usage of these packets is +designed so that they need not be delivered reliably. + +Retransmission of a packet is stopped by a signal from the receiver to +the sender called a 2receipt*. A receipt contains a 2packet +number*, and indicates that all controlled packets with a packet +number less than or equal (modulo 65536) to that number have been +successfully received, and therefore need not be retransmitted any +more. A receipt does not indicate that these packets have been +successfully delivered to the destination user process; it simply +indicates that they have successfully arrived in the destination host, +and are guaranteed to be there when the user process asks for them. + +There is another signal from the receiver to the sender, called +an 2acknowledgement*. An acknowledgement also contains a packet +number, and indicates that all controlled packets with a packet number +less than or equal (modulo 65536) to that number have been delivered +to the destination user process. This is used to implement flow-control. +Note that acknowledgement of a packet implies receipt of that packet. +In fact, if the receiving process does not fall behind, +explicit receipts need not be sent. + +The purpose of flow-control is to match the speeds of the sending and +receiving processes. The extremes to be avoided are, on the one hand, +too small a "buffer size" causing the data transmission rate to be slower +than it could be, and on the other hand, large numbers of packets piling +up in the network because the sender is sending faster than the receiver +is receiving. It is also necessary to be aware that receipts and +acknowledgements must be transmitted through the network, and hence +have an associated cost. + +Chaosnet flow-control operates by controlling the number of packets "in +the network". These are packets which have been emitted by the sending +user process, but have not been acknowledged. We define a 2window* +into the set of packet numbers. The beginning of this window is the +first packet number which has not been acknowledged, and the width of +the window is a fixed number established when the connection is opened. +The sending process is only allowed to emit packets whose packet +numbers lie within the window. Once it has emitted all of the packets +in the window, the window is said to be full. Thus, the size of the +window is the "buffer size" for the connection, and is the maximum +number of packets that may need to be buffered inside an NCP (sending +or receiving). Acknowledgements move the window, making it not full, +and allowing the sending process to emit additional packets. + +We do not receipt and acknowledge every single controlled packet that is transmitted +through a connection, since that would double or triple the number of packets +sent through the network to move a given amount of data. Instead we batch +the receipts and acknowledgements. But if acknowledgements are not sent sufficiently +often, the data will not flow smoothly, because the window will often appear +full to the sender when it is not, and if receipts are not sent sufficiently +often, there will be unnecessary retransmissions. + +Whenever a packet is sent through a connection, an acknowledgement for the +reverse direction of that connection is "piggy-backed" onto it, using the +Acknowledgement field in the packet header. For interactive applications, +where there is much traffic in both directions, this provides all the necessary +acknowledgement and receipting with no need to send any extra packets through +the network. + +When this does not suffice, STS (status) packets are generated to carry receipts +and acknowledgements. STS packets are uncontrolled, since they are part of the +controlled-packets mechanism. If a STS packet is duplicated, it does no harm. +If a STS packet is lost, mechanisms exist which will cause a replacement to +be generated later. A STS packet carries separate receipt and acknowledgement +packet numbers. + +When a user process reads a packet from the network, if the number of packets which +should have been acknowledged but have not been is more than 1/3 the window size, +a STS is generated to acknowledge them. Thus the preferred batch size for acknowledgement +is 1/3 the window size. The advantage of this size is that if one STS is lost, +another will be generated before the window fills up (at the 2/3 point). + +When a packet is received with the same packet number as one which has already been +successfully received, this is evidence of unnecessary retransmission, and +a STS is generated to carry a receipt back to the sender. If this STS is lost, +the next retransmission will stimulate another one. Thus receipts are normally +implied by acknowledgements, and only sent separately when there is evidence of +unnecessary retransmission. + +Retransmission consists of sending all unreceipted controlled packets, +except those that were last sent very recently (within 1/30'th of a +second in ITS.) Retransmission occurs every 1/2 second. This interval +is somewhat arbitrary, but should be close to the response time of the +systems involved. Retransmission also occurs in response to a STS +packet, so that a receiver may cause a faster retranmission rate than +twice a second if it so desires. This should never cause useless +retransmission, since STS carries a receipt, and +very-recently-transmitted packets, which might still be in transit +through the network, are not retransmitted. + +Another operation is 2probing*, which consists of sending a SNS +packet, in the hope of eliciting either a STS or a LOS, depending on +whether the other side believes the connection exists. Probing is used +periodically as a way of testing that the connection is still open, and +also serves as a way to get STS packets retransmitted as a hedge +against the loss of an acknowledgement, which could otherwise stymie +the connection. SNS packets are uncontrolled. + +We probe every five seconds on connections which have unacknowledged +packets outstanding (a non-empty window), and on connections which have +not received any packets for one minute. If a connection receives no +packets for 1 1/2 minutes, this means that at least 5 probes have been +ignored, and the connection is declared to be broken; either the remote +host is down or there is no viable path through the network between the +two hosts. + +The receiver can generate "spontaneous" STS's, to stimulate +retransmission and keep things moving on fast devices with insufficient +buffering for 1/2 second worth of packets. This provides a way for the +receiver to speed up the retransmission timeout in the sender, and to +make sure that acknowledges are happening often enough. + +Note that the network still functions if either or both parties to a +connection ignore the window. The window is simply an improver of +efficiency. Receipts have the same property. This allows very small +implementations to be compatible with the same protocol, which is +useful for applications such as bootstrapping through the network. + +It would be possible to have dynamic adjustment of the window size in +response to observed behavior. The STS packet includes the window size +so that changes to it can be communicated. However, this has not been found +necessary in practice. Each higher-level protocol has a standard +pre-determined window size, which it establishes when it first opens a +connection. + +This scheme for flow-control and error-control is based on several assumptions. +It is assumed that the underlying transmission media have their own checking, +so that they discard all damaged packets, making packet checksums unnecessary +at the protocol level. The transit time through the network is assumed to be +fast, so that a fairly-small retransmission interval is practical, and +negative acknowledgements are not necessary. It is assumed that no reformatting +of packets occurs inside the network, so that flow-control and error-control +can operate on a packet basis rather than a byte basis. + +.chapter Software Protocol--Details + +In the following sections, each of the packet 2Opcodes* +and the use of that packet type in the protocol is described. +Opcodes are given as an octal number, a three-letter code, and a name. + +Unless otherwise specified, the use of the fields in the packet header +is as follows. The source and destination address and index denote +the two ends of the connection; when an end does not exist that index is zero. +The opcode, byte count, and forwarding count fields have no variations. +The packet number field contains sequential numbers in controlled packets; +in uncontrolled packets it contains the same number as the next controlled +packet will contain. The acknowledgement field contains the packet +number of the last packet seen by the user. + +.section Connection Establishment + +.packet 1 RFC Request For Connection +All connections are initiated by the transmission of an RFC from the user to the server. +The data field of the packet contains the contact name. The contact name can +be delimited by a space character and followed by arbitrary arguments to the server. +The destination index field of an RFC contains 0 since the destination index is +not known yet. + +RFC is a controlled packet; it is retransmitted until some sort of response +is received. Because RFC's are not sent over real connections, a special +way of detecting and discarding duplicates is required. When an NCP receives +an RFC packet, it should check all pending RFC's and all connections which are +in the Open or RFC-received state (see below), +to see if the source address and index match; if so, the RFC is a duplicate +and should be discarded. + +RFC's are given to the user process, so that it can look at the contact name +and any arguments that may be present. + +.packet 12 LSN Listen +A server process informs the local NCP of the contact name to which it is +listening by sending a LSN packet, with the contact name in the data field. +This packet is never transmitted anywhere through the network. It simply +serves as a convenient buffer to hold the server's contact name. When an RFC +and a LSN containing the same contact name meet, the LSN is discarded and +the RFC is given to the server, putting its connection into the RFC-received +state (see below). The server reads the RFC and decides whether or not to open the connection. + +.packet 2 OPN Open connection +OPN is the usual positive response to RFC. The source index field conveys the +server's index number to the user; the user's index number was conveyed in the RFC. +The data field of OPN is the same as that of STS; it serves mainly to convey +the server's window-size to the user. The Acknowledgement field of the OPN +acknowledges the RFC so that it will no longer be retransmitted. + +OPN is a controlled packet; it is retransmitted until it is acknowledged with STS. +This STS conveys the user's window-size to the server and permits the server +to start transmitting data. The exchange of an OPN and a STS tells each side +of the connection that the other side believes the connection is open; once this +has happened data may begin to flow through the connection. + +Duplicate OPN packets are detected in a special way; if an OPN is received +for a connection which is not in the RFC-sent state, it is discarded. + + +.packet 3 CLS Close connection +CLS is the negative response to RFC. It indicates that no server was +listening to the contact name, and one couldn't be created, or for +some reason the server didn't feel like accepting this request for a +connection, or the destination NCP was unable to complete the +connection (e.g. connection table full.) + +CLS is also used to close a connection after it has been open for a +while. Any data packets in transit may be lost. Protocols which +require a reliable end-of-data indication should use the mechanism for +that (described below) before sending CLS. + +The data field of a CLS contains a character-string explanation of the +reason for closing, intended to be returned to a user as an error message. + +CLS is an uncontrolled packet, so that the program which sends it may +go away immediately afterwards, leaving nothing to retransmit the +CLS. Since there is no error recovery or retransmission mechanism for +CLS, the use of CLS is necessarily optional; a process could simply +stop responding to its connection. However, it is desirable to send a +CLS when possible to provide an error message for the user. + +.packet 4 FWD Forward a request for connection +This is a response to RFC which indicates that the desired service is +not available from the process contacted, but may be available at a +possibly-different contact name at a possibly-different host. The data +field contains the new contact name and the Acknowledgement +field--exceptionally--contains the new host number. The issuer of the +RFC should issue another RFC to that address. FWD is an uncontrolled +packet; if it is lost in the network, the retransmission of the RFC +will presumably stimulate an identical FWD. + +.packet 5 ANS Answer to a simple transaction +This is another kind of response to RFC. The data field contains +the entirety of the response, and no connection is established. +ANS is an uncontrolled packet; if it is lost in the network, the +retransmission of the RFC will presumably stimulate an identical ANS. + +There are several connection-initiation protocols implemented with +these packet types. + +A 2regular connection* is initiated by an RFC containing a contact name, +transmitted from user to server. If the server process does not +already exist, the RFC may provoke the operating system on the server host +into creating the server process. The server process listens to +the same contact name, which causes the RFC to be directed +to that process. The server returns an OPN to the user, which responds +with a STS. These three packets convey the source and destination addresses, +indices, initial packet numbers, and window sizes between the two +NCP's. In addition a character-string argument can be conveyed from +the user to the server in the RFC. The user process may begin +transmitting data when it sees the OPN. The server process may begin +transmitting data when it sees the STS. These rules ensure that data +packets cannot arrive at a receiver before it knows and agrees that the +connection is open. Retransmission of the RFC and the OPN provides +reliability in the face of lost packets. + +A 2refusal* is initiated by an RFC in the same way, but +the server returns CLS rather than OPN. The data field of the CLS contains +the reason for refusal to connect. + +A 2forwarded connection* is initiated by an RFC in the same way, +but the server returns a FWD, telling the user another place to look +for the desired service. + +A 2simple transaction* is initiated by an RFC from user to server, and completed +by an ANS from server to user. Since a full connection is not established +and the reliable-transmission mechanism of connections is not used, the user +process cannot be sure how many copies of the RFC the server saw, and the server +process cannot be sure that its answer got back to the user. This means that +simple transactions should not be used for applications where it is important +to know whether the transaction was really completed, nor for applications +in which repeating the same query might produce a different answer. +Simple transactions are a simple efficient mechanism for applications such as extracting +a small piece of information (e.g. the time of day) from a central data-base. + +.section Status + +.packet 7 STS Status + +STS is an uncontrolled packet which is used to convey status +information between NCPs. The Acknowledgement field in the packet +header contains an acknowledgement, that is, the packet number of the +last packet given to the receiving user process. The first 16-bit byte +in the data field contains a receipt, that is, a packet number such +that all controlled packets up to and including that one have been +successfully received by the NCP. The second 16-bit byte in the data +field contains the window size. The byte count is presently always 4. +This will change if the protocol is revised to add additional items to +the STS packet. + +.packet 6 SNS Sense status + +SNS is an uncontrolled packet whose sole purpose is to cause the +other end of the connection to send back a STS. This is used by +the 2probing* mechanism described above. + +.packet 11 LOS Lossage + +LOS is an uncontrolled packet which is used by one NCP to inform +another of an error. +'c Also an uncontrolled poet, he said? +The data field contains a character-string explanation of the problem. +The source and destination addresses and indices are simply the +destination and source addresses and indices, respectively, of the +erroneous packet, and do not necessarily correspond to a connection. +When an NCP receives a LOS whose destination corresponds to an existent +connection and whose source corresponds to the supposed other end of +that connection, it 2breaks* the connection and makes the data field +of the LOS available to the user as an error message. Other LOS's, +that don't correspond to connections, are simply ignored. + +LOS is sent in response to situations such as: arrival of a data packet +or a SNS for a connection that does not exist or is not open, arrival +of a packet from the wrong source for its destination, arrival of a packet +containing an undefined opcode or too large a byte count, etc. + +LOS's are given to the user so that it may read the error message. + +.section Data +200-377,UNC + +.packet 200-277 DAT 8-bit Data + +Opcodes 200 through 277 (octal) are controlled packets with user data +in 8-bit bytes in the data field. The NCP treats all 64 of these opcodes +identically. Some higher-level protocols use the opcodes for their +own purposes. The standard default opcode is 200. + +.packet 300-377 DAT 16-bit Data + +Opcodes 300 through 377 (octal) are controlled packets with user data +in 16-bit bytes in the data field. The NCP treats all 64 of these +opcodes identically. The standard default opcode is 300. + +.packet 15 UNC Uncontrolled Data + +This is an uncontrolled packet with user data in 8-bit bytes in +the data field. It exists so that user-level programs may bypass +the flow-control mechanism of the Chaosnet protocol. Note that +the NCP is free to discard these packets at any time, since they +are uncontrolled. Since UNC's are not subject to flow control, discarding +may be necessary to avoid running out of buffers. + +.section End-of-Data + +.packet 14 EOF End of File + +EOF is a controlled packet which serves as a "logical end of data" mark +in the packet stream. When the user program is ignoring packets and +treating a Chaosnet connection as a conventional byte-stream I/O device, +the NCP uses the EOF packet to convey the notion of conventional end-of-file +from one end of the connection to the other. When the user program +is working at the packet level, it may transmit and receive EOF's. + +EOF packets are also used in the following protocol which is the recommended +way to reliably determine that all data have been transferred before +closing a connection (in applications where that is an important +consideration). + +The important issue is that neither side may send a CLS until both sides +are sure that each side has received all the data. After sending all +the data it is going to send, each process waits for the last data packet +to be acknowledged. It then sends an EOF. As soon as it receives an EOF, +it may close, since that EOF indicates that the other process has seen +the acknowledgement for the last data packet it sent. Note that if EOF +is also being used to mark the end of the data, it is necessary to use two +EOFs, the first one taking the place of the "last data packet" above, +and the second one indicating readiness to close. + +.section Low-level + +.packet 13 MNT Maintenance + +MNT is a special packet type reserved for the use of network +maintenance programs. Normal NCPs should simply discard any MNT +packets they receive. Except for the opcode, the packet header +need not conform to the normal protocol. + +.packet 10 RUT Routing Information + +RUT is a special packet type broadcast by bridges to inform other +nodes of the bridge's ability to forward packets between subnets. +The source address is the network address of the bridge on the +subnet on which the RUT was broadcast. The destination address is zero. +The byte count is a multiple of 4, and the data field contains a series +of pairs of 16-bit bytes: a subnet number and the "cost" of getting to +that subnet via this bridge. The packet number and acknowledgement +fields are not used and should contain zero. + +.section Connection States + +A user process gets to Chaosnet by means of a capability or channel +(dependent on the host operating system) which corresponds to one +end of a connection. Associated with this channel are a number of +buffers containing controlled packets output by the user and not +yet receipted, and packets received from the network but +not yet read by the user; some of these incoming packets are +in-order by packet number and hence may be read by the user, while +others are out of order and cannot be read until packets earlier +in the stream have been received. + +Also associated with the channel is a state, usually called the +2connection state*. Full understanding of these states depends +on the descriptions of packet-types below. The state can be one of: +.table 2 +.item Open +The connection exists and data may be transferred. +.item Closed +The channel does not have an associated connection. Either it +never had one or it has received or transmitted a CLS packet, +which destroyed the connection. +.item Listening +The channel does not have an associated connection, but it has a contact +name (usually contained in a LSN packet) for which it is listening. +.item RFC Received +A 2Listening* channel enters this state when an RFC arrives. +It can become 2Open* if the user process 2accepts* the request. +.item RFC Sent +The user has transmitted an RFC. The state will change to 2Open* or +2Closed* when the reply to the RFC comes back. +.item Lost +The connection has been broken by receiving a LOS packet. +.item Incomplete Transmission +The connection has been broken because the other end has ceased +to transmit and to respond to SNS. Either the network or the foreign +host is down. +.end_table + +.chapter Higher-Level Protocols + +.section Status + +All network nodes, even bridges, are required to answer RFC's +with contact name 7STATUS*, returning an ANS packet in a simple transaction. +This protocol is primarily used for network maintenance. +The answer to a 7STATUS* request should be generated by +the Network Control Program, rather than by starting up a server +process, in order to provide rapid response. + +The first 32 bytes of the ANS contain the name of the node, padded on +the right with bytes containing octal 200. The rest of the packet +contains blocks of information expressed in 16-bit words, low byte +first (pdp11/Lisp machine style). The first word in a block is its +identification. The second word is the number of words to follow. + +If the identification is less than 400 octal, it is a subnet number and +this block contains information on the node's direct connection to +that subnet. Values larger than 400 are reserved for future use. + +These are the 16-bit words in a subnet information block. All items +are optional, according to the count field, and extra items not defined +here may be present and should be ignored. Note that because the counts +are 16-bit unsigned numbers, they may overflow. +.table 1 +.item word 0 +The subnet number. +.item word 1 +The number of words to follow, usually 8. +.item word 2 +The number of packets received from this subnet. +.item word 3 +The number of packets transmitted to this subnet. +.item word 4 +The number of transmissions to this subnet aborted by collisions or by receiver busy. +.item word 5 +The number of incoming packets from this subnet lost because the +node had not yet read a previous packet out of the interface. +.item word 6 +The number of incoming packets from this subnet with CRC errors. +These were either transmitted wrong or damaged in transmission. +.item word 7 +The number of incoming packets from this subnet which had no CRC +error when received, but did have an error after being read out of +the packet buffer. This error indicates either a problem with the +packet buffer or an incorrect packet length. +.item word 8 +The number of incoming packets from this subnet which were rejected +due to incorrect length (typically not a multiple of 16 bits). +.item word 9 +The number of incoming packets from this subnet rejected for +other reasons (e.g. too short to contain a header, garbage byte-count, +forwarded too many times.) +.end_table + +.section Pulsar + +For network maintenance purposes, certain network nodes support a +simple transaction with contact name 7PULSAR*, which controls a +"pulsar" feature. This feature periodically transmits a short packet +addressed to a non-existent host, which can be used to test and adjust +cable transceivers. + +The returned ANS contains a single character, which is a digit. A 0 +means that the pulsar is turned off. Another digit indicates the number +of sixtieths of a second between pulses. Sending an RFC with a digit +as an argument sets the state of the pulsar to that digit, and returns +an ANS containing the new state. + +.section Telnet and Supdup + +The Telnet and Supdup protocols of the Arpanet [ref.] exist in +identical form on the Chaosnet. These protocols allow access to a +computer system, from another network node, as an interactive terminal. + +The contact names are 7TELNET* and 7SUPDUP*. The only difference, +other than the way the connection is first established, is that the +INR/INS attention-getting feature of the Arpanet has no counterpart on +the Chaosnet. It is not necessary with the type of hosts attached to +Chaosnet. The direct borrowing of the Telnet and Supdup protocols was +eased by their use of 8-bit byte streams and only a single connection. + +.section Time + +The Time protocol of the Arpanet [ref.] exists on Chaosnet as a simple +transaction. An RFC to contact name 7TIME* evokes an ANS containing +the number of seconds since midnight Greenwich Mean Time, Jan 1, 1900 +as a 32-bit number in four 8-bit bytes, least-significant byte first. +Some operating systems use this protocol to find out the date and time +when they first come up. + +.section Arpanet Gateway + +This protocol allows a Chaosnet host to access almost any service on +the Arpanet. The gateway server runs on each ITS host that is connected +to both networks. It creates an Arpanet connection and a Chaosnet connection +and forwards data bytes from one to the other. It also provides for a one-way +auxiliary connection, used for the data connection of the Arpanet File Transfer Protocol. + +The RFC packet contains a contact name of 7ARPA*, a space, the name +of the Arpanet host to be connected to, optionally followed by a space +and the contact-socket number in octal, which defaults to 1 if omitted. +The Arpanet Initial Connection Protocol is used to establish a bi-directional +8-bit connection. + +If a data packet with opcode 201 (octal) is received, an Arpanet INS +signal will be transmitted. Any data bytes in this packet are transmitted normally. + +If a data packet with opcode 210 (octal) is received, an auxiliary +connection on each network is opened. The first eight data bytes are +the Chaosnet contact name for the auxiliary connection; the user should +send an RFC with this name to the server. The next four data bytes are the +Arpanet socket number to be connected to, in the wrong order, most-significant +byte first. The byte-size of the auxiliary connection is 8 bits. + +The normal closing of an Arpanet connection corresponds to an EOF packet. +Closing due to an error corresponds to a CLS packet. + +.chapter Hardware Programming Documentation + +This section describes the Unibus version of the Chaosnet interface, +which attaches to pdp11s and Lisp machines. The interface contains +one buffer which holds a received packet and a second buffer which +holds a packet to be transmitted. Packets are moved between +these buffers and the computer under program control. Direct +memory access (DMA) is not used; the small gain in performance was +not thought to be worth the extra hardware complexity. The usual +performance penalty of programmed I/O is not incurred since the +packet buffers can transfer data at the full speed of the computer and +neither busy waiting nor multiple interrupts are required. + +To transmit a packet, successive 16-bit words of the packet are written +into the outgoing packet buffer. The last word to be written is the +cable address of the destination of the packet, or 0 to broadcast it. +The hardware is then told to initiate transmission. It waits until +the cable is not busy and this node's turn to transmit arrives, then +shifts the packet out onto the cable. At the completion of transmission +transmit-done is set and the computer is interrupted. If transmission +is aborted by a collision, transmit-done and transmit-abort are set +and the computer is interrupted. As the packet is written into the +outgoing packet buffer, a 16-bit cyclic redundancy checksum is computed +by the hardware. This checksum is transmitted with the packet and checked +by the receiver. + +To receive a packet, the clear-receiver bit is asserted by the program. +The next packet on the cable which is addressed to this node, or is broadcast, +will be stored into the incoming packet buffer. After the packet has +been stored, the computer is interrupted. The packet buffer will then +not be changed until the next clear-receiver operation is performed, +giving the computer a chance to read out the packet. If a packet appears +on the cable addressed to this node while the incoming packet buffer +is busy, a collision is simulated so as to abort the transmission. +As a packet is stored into the incoming packet buffer, the 16-bit +cyclic redundancy checksum is checked, and it is checked again as +the packet is read out of the packet buffer. This provides full checking +for errors in the network and in the packet buffers. + +The standard interrupt-vector address for the Chaosnet interface +is 270. The standard interrupt priority level is 5. +The standard Unibus address is 764140. These are the +device registers: +.table 1 0 500 +.item 764140 2Command/Status Register* +This register contains a number of bits, in the usual pdp11 style. +All read/write bits are initialized to zero on power-up. +Identified by their masks, these are: +.table 7 0 600 +.item " 1" +Transmit Busy (read only). +.item " 2" +Loop Back (read/write). If this bit is 1, the cable and transceiver are +not used and the interface is looped back to itself. This is for maintenance. +.item " 4" +Spy (read/write). If this bit is 1, the interface will receive all packets +regardless of their destination. This is for maintenance and network monitoring. +.item " 10" +Clear Receiver (write only). Writing a 1 into this bit clears Receive Done +and enables the receiver to receive another packet. +.item " 20" +Receive Interrupt Enable (read/write). If Receive Done and Receive Interrupt Enable +are both 1, the computer is interrupted. +.item " 40" +Transmit Interrupt Enable (read/write). If Transmit Done and Transmit Interrupt Enable +are both 1, the computer is interrupted. +.item " 100" +Transmit Abort (read only). This bit is 1 if the last transmission was aborted +by a collision. +.item " 200" +Transmit Done (read only). This bit is set to 1 when a transmission is +completed or aborted, and cleared to 0 when a word is written into the outgoing +packet buffer. +.item " 400" +Clear Transmitter (write only). Writing a 1 into this bit stops the transmitter +and sets Transmit Done. +.item " 17000" +Lost Count (read only). These 4 bits contain a count of the number of packets +which would have been received if the incoming packet buffer had not been busy. +Setting Clear Receiver resets the lost count to 0. +.item " 20000" +Reset (write only). Writing a 1 into this bit completely resets the interface, +just as at power up and Unibus Initialize. +.item " 40000" +CRC Error (read only). If this bit is 1 the receiver's cyclic redundancy +checksum indicates an error. This bit is only valid at two times: when the incoming packet +buffer contains a fresh packet, and when the packet has been completely read out +of the packet buffer. +.item "100000" +Receive Done (read only). A 1 in this bit indicates that the incoming packet +buffer contains a packet. +.end_table + +.item 764142 2My Address* (read only) +Reading this location returns the network address of this interface +(which is contained in a set of DIP switches on the board). +.item 764142 2Write Buffer* (write only) +Writing this location writes a word into the outgoing packet buffer. +The last word written is the destination address. +.item 764144 2Read Buffer* (read only) +Reading this location reads a word from the incoming packet buffer. +The last three words read are the destination address, the source address, +and the checksum. +.item 764146 2Bit Count* (read only) +This location contains the number of bits in the incoming packet buffer, minus one. +After the whole packet has been read out, it will contain 7777 (a 12-bit minus-one). +.item 764152 2Start Transmission* (read only) +Reading this location initiates transmission of the packet in the +outgoing packet buffer. The value read is the network address of this interface. +This method for starting transmission may seem strange, but it makes it +easier for the hardware to get the source address into the packet. +.end_table + +.chapter Using Chaosnet in ITS + +.chapter Using Chaosnet in TOPS-20 + +.chapter Using Chaosnet in Lisp Machine + +.chapter Design Issues + +.chapter Writing an NCP + + + + + + + +--- gubbish --- +>The Protocol + + The purpose of the basic software protocol of the Chaos network is +to allow high-speed communication among processes on different machines, +with no undetected transmission errors. The speed is to be comparable with +an inexpensive magnetic tape drive (30000 characters per second), or about +10 times the speed of the Arpanet. It is important to design out +bottlenecks such as are found in the Arpanet, e.g. the shared control-link +and the need to acknowledge each message before the next message can be +sent. The protocol must be simple, for the sake of reliability and to +allow its use by modest computer systems. A full Chaos Network Control +Program is just about half the size of an Arpanet NCP on the same machine, +and the protocol allows low-performance implementations to omit some +features. + + The main facility provided is called a "connection". It is a +full-duplex channel through which packets may be sent between two user +processes. Users usually do not see packet boundaries and regard the two +streams of packets as streams of 8-bit bytes. The network undertakes never +to garble, lose, duplicate, or resequence the streams; in the event of a +serious error it may break the connection off entirely. + + On top of the connection facility user processes build other +facilities, such as file access, interactive terminal connections, and +data in byte sizes other than 8 bits. + + In addition to communication, the protocol provides a way by which +prospective communicants may get in touch with each other, called +"contacting" (or rendezvous). We choose to use an asymmetrical model and +call one process the "user" and the other the "server". The server has +some "contact name" to which it "listens". The user requests its operating +system to connect it to a specified contact name at a specified host. If a +process at that host is listening to that contact name, the two are +connected. If no one is listening to that contact name, a default handler +can look up the contact name in a table and create a server process running +the appropriate program, or refuse the request for a connection if the +name is not in the table. + + Discovering which host to connect to to obtain a given service is +an issue for higher-level protocols. It is currently not dealt with at all +(that is, there is a table of host names and numbers and the user has to +supply the name.) + + Once the connection has been established, there is no more need for +the contact name, and it is discarded. Indeed, often the contact name is +simply the name of a network protocol (such as "telnet") and several users +may want to have connections to that service at the same time, so contact +names must be "reusable." + + +[Explain about sites, NCPs, processes, indices, addresses] + +>>Packet Format and Types +Basic description of each packet type, details left for later +(Explicitly indicate that there are forward references to following section.) + +>>Flow and Error Control Technique +[This may be largely copyable from CHAORD] +Windows +Retransmission +Acknowledgement +Receipt +Probe + +>>Routing +[non-existent for now] + +>>Higher-level protocols (briefly) + +>>Robustness +[??] + + The Chaos network is robust because it is immune to momentary +failures and because it is completely distributed. The +collision-detection and packet CRC checkwords detect momentary +failures; the acknowledgement mechanism provides positive confirmation +that there was no failure. The retransmission protocols enable +recovery from all momentary failures. Certain permanent failures can +bring down the whole network, but these are few because the hardware is +distributed. Also permanent failures are much easier to find and fix +than momentary non-repeatable ones. + + The ether itself can fail, for instance someone might saw the +cable in half accidentally, but it is not expected to fail often since +it is a passive device. If one interface should decide to transmit +continually, it can jam the ether so that no one else can get a message +through. There are defensive circuits which have to fail for this to +happen. In fact, it did happen once; an interface got into a state +where it was continuously sending alternating 1's and 0's, and failed +to decide that the end of a packet had been reached so that it was time +to stop. The design has since been revised so that this particular +failure cannot happen again. + +>ITS System Calls +[??] [Summary at least, even if not a complete programmer's specification.] + +>A Simplified NCP + +>>Variables + +>>Packet Receive Process + +>>Packet Send Process + +>>Clock Process + +>>User Receive Process + +>>User Send Process + +>>Suppressed Details + +>Comparison with the Arpanet +Robustness, Performance/Overhead, Simplicity +[There's some of this up above] + +>Comparison with TCP +[would be nice] + +>Experience/Implementation/Cost +[not yet?] + +.c Local Modes: +.c ...0: ^:i*{ 6i 60.i .-2,. { +.c ...1: ^:i*{ 6i 61.i .-2,. { +.c ...2: ^:i*{ 6i 62.i .-2,. { +.c ...3: ^:i*{ 6i 63.i .-2,. { +.c ...7: ^:i*{ 6i 67.i .-2,. { +.c ...*: ^:i*{ 6i i* .-2,. { f(u...:) +.c ...:: ^:i*{ 6i i* .-2,. { +.c End: +.eof diff --git a/doc/moon/amber.bolio b/doc/moon/amber.bolio new file mode 100644 index 00000000..e11942bb --- /dev/null +++ b/doc/moon/amber.bolio @@ -0,0 +1,30 @@ +;; BOLIO init file for Amber paper -*-LISP-*- + +;; Not much to say, yet... + +(PAPER-STYLE '|The Chaosnet *** DRAFT ***|) + +;;; .packet opcode 3-letter-code long-name +(DEFPROP PACKET PACKET-REQUEST REQUEST) + +(DEFUN PACKET-REQUEST () + (CHECK-ENV 'TEXT 'PACKET) + (OR (NEED-SPACE 200.) ;1 inch + (OUTPUT-LEADING 50.)) ;1/4 inch + (CHECK-FONT-STATUS 1) + (SETQ CUR-HPOS 0) + (LET ((NUMERIC-CODE (GET-WORD-STRING)) + (3LETTER-CODE (GET-WORD-STRING)) + (FULL-NAME (GET-LINE-STRING)) + (JIN-CUR-FONT 7) + (TAB-STRING (STRING '| |))) + (SET-HPOS LEFT-MARGIN) + (PUT-STRING-FLUSH-LEFT NUMERIC-CODE) + (OUTPUT-NOFILL-STRING TAB-STRING) + (PUT-STRING-FLUSH-LEFT 3LETTER-CODE) + (OUTPUT-NOFILL-STRING TAB-STRING) + (SETQ JIN-CUR-FONT 1) + (PUT-STRING-FLUSH-LEFT FULL-NAME)) + (LINE-ADVANCE) + (SETQ BEGIN-NEW-PARAGRAPH T) + (OUTPUT-LEADING 25.)) ;1/8 inch