Date: Sat, 26 May 90 18:38:46 EDT From: "Pandora B. Berman" Sender: CENT5@MC.LCS.MIT.EDU Subject: bouncing mail To: AVIATION-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, PAGANISM-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, SCHEME-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, SUBGENIUS-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU cc: SCA-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <671367.900526.CENT5@MC.LCS.MIT.EDU> fyi, this is the administrative msg we have installed to go out with the next SCA digest. if your digests have also had bouncing mail as described, you might want to place similar msgs to go out with your digests. the KS is currently hard to reach, so if you want to place such a msg but can't reach the KS to do so, send your msg to me and i will install it appropriately. ---------- Modern-world interrupt for the technically interested: The name MC.LCS.MIT.EDU was moved yesterday from the KS10 running ITS to a Microvax running some version or another of unix (don't ask me which). The KS10 wasn't on the Internet; the Microvax is. For a brief period during the move there was confusion about which host the name belonged to, which caused some mail to the digest to bounce. We believe that all such mail actually bounced to the -Request list. All the mail that bounced in this fashion to the -Request list has now been dropped back into the SCA incoming mailbox for digestification. If you sent mail to the digest and it returned to you complaining that MC was an unknown host or SCA@MC was an unknown address, please try again. Otherwise, please just wait a bit, and your missive should appear -- we're a talkative lot, and the digestifier is running a bit behind our volubility. At this moment the digest is still being produced on the KS10, which will continue to run for a few more weeks under an assumed name. This works by magic -- don't ask. A new improved digestifier which will run on unix is currently being written and should be ready soon. At that point the digestification process will move to the Microvax, and there may be some disruption as we shake out the bugs. Watch this space for updates. We now return you to news of the Current Middle Ages. SCA-Request@MC.LCS.MIT.EDU  Date: Mon, 14 May 90 00:03:24 EDT From: "Pandora B. Berman" Subject: MC going down the tubes To: SCA-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, PAGANISM-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, SCHEME-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, AVIATION-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, SUBGENIUS-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU cc: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <667947.900514.CENT@MC.LCS.MIT.EDU> As you may have heard, the MC ITS machine is going away soon. Regular mail will be turned off 17 May; at that point the name MC.LCS.MIT.EDU will move to some unix box belonging to LCS, and the ITS machine will answer only to the name OLD-MC.LCS.MIT.EDU. By the 17th we expect to have moved all ordinary mailing lists from the old MC to the new one, so that transfer should be fairly transparent. However, we won't move the digests then. Instead, we will set up on the new MC forwarding pointers to direct the mail for the digests to their mailboxes on the old MC. The ITS digestifier will continue to use those inboxes to produce the digests for another sevral days. The reason we are putting off moving the digests is that, while we have been given digestifiers for unix, we have also been told they are not very effective. A noble volunteer is working on a new digestifier, writing in GNU Emacs Lisp, which we expect to be more useful than the currently available programs, and which we have every hope will be ready before the old MC is physically turned off on 25 May. If it isn't, we will temporarily use whichever of the already-available digestifiers looks least unreasonable. Since the situation is still very fluid, we can't guarantee that the digests will work during this transitional period. You may wish to use the Administrivia feature of the ITS digestifier to warn your list members that things may not run smoothly during the next few weeks.  Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU; 10 May 90 08:50:41 EDT Received: from umich.edu by mintaka.lcs.mit.edu id aa17863; 10 May 90 8:40 EDT Received: from ummts.cc.umich.edu by umich.edu (5.61/1123-1.0) id AA28352; Thu, 10 May 90 08:40:03 -0400 Date: Thu, 10 May 90 08:38:48 EDT X-Modified: Modified by sender Thu, 10 May 90 08:39:26 EDT and reposted. From: Pat_McGregor@um.cc.umich.edu To: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.lcs.mit.edu, CENT@MC.lcs.mit.edu, EXPOTECH@applelink.apple.com Cc: SCA-REQUEST%MC.LCS.MIT.EDU@MINTAKA.lcs.mit.edu, Mailing-List-Administrator@um.cc.umich.edu Message-Id: <6172788@um.cc.umich.edu> Subject: Sca digest address problem Unto the Lady Eilonwy, redoubtable guardian of the Rialto, Ladie Catherine Aimee and Gwynneth, and unto the estimable Mukra, my coworker, come greetings from Siobhan Medhbh O'Roarke, with a foot in each World. Lady Catherine-Aimee's registration for the digest has been removed as of 5 minutes ago. Please add: EXPOTECH@applelink.apple.com in to the rolls ethereal immediately lest they become despondent over their lack of connectedness! Regards, siobhan  Date: Thu, 10 May 90 02:40:40 EDT From: "Pandora B. Berman" Subject: Sca digest address problem To: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, "EXPOTECH@applelink.apple.com"@MINTAKA.LCS.MIT.EDU, "postmaster@applelink.apple.com"@MINTAKA.LCS.MIT.EDU cc: SCA-REQUEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, mailing-list-administrator@UM.CC.UMICH.EDU, smor@UM.CC.UMICH.EDU Message-ID: <666791.900510.CENT@MC.LCS.MIT.EDU> Subject: Sca digest address problem To: SCA-REQUEST@MC.lcs.mit.edu From: "Expotech, Aimee Moran,VCA" Date: 19 Apr 90 13:38 GMT Milords, Justin du Coeur indicated that you are the sysops for the Rialto, and to lay our problem at your feet... We have been receiving the Sca DIgests regularly for a couple of months now. Last Sunday, April 15, AppleLink changed its gates with Internet and Bitnet, so that although our personal messages are going out and coming in okay, suddenly we are not receiving the digests. Could you please check this out for us? Our internet address is: Expotech@Applelink.apple.com Thank you very kindly for your assistance. Catherine-Aimee leMoyne Gwynnyd of York Date: Thu, 19 Apr 90 21:56:14 EDT From: MarkA_Davis@um.cc.umich.edu To: postmaster@applelink.apple.com Cc: sca-request@mc.lcs.mit.edu, Mailing-List-Administrator@um.cc.umich.edu Subject: Mailing lists The list administrator of the Society for Creative Anachronism mailing list is having trouble sending to someone at your host. Could you please investigate? Also, there's really no need for the list to stop by the University of Michigan on its way to apple. If it's agreeable to everyone else, I'd like to remove EXPOTECH from our exploder and have it added to the main list at MIT. Mark Davis, mailing-list-administrator@um.cc.umich.edu, LISTADM@UMICHUM Cc: SMOR@um.cc.umich.edu, "Expotech, Aimee Moran,VCA" Subject: Digests lost... To: SCA-REQUEST@mc.lcs.mit.edu From: "Expotech, Aimee Moran,VCA" Date: 09 May 90 13:37 GMT Hello, SCA-sysops! Lamentably we are still having fits with our AppleLink-Internet address. We began getting the Digest again last week (we got 678, 679, and 680) and then it stopped again. I am in communication with the AppleLink Internet postmaster, and we are pursuing the glitch to see if it originates at the gate. Would you please, one more time, check our address and make sure it is "expotech@applelink.apple.com"? Oddly, we are getting other mail which is addressed that way, but not the Digest. Obviously, we miss the digest, and wish also to be sure we're not missing other messages. I am *greatly* appreciative of your efforts in this matter. Sincerely, Catherine-Aimee leMoyne All and sundry: We apologize about the delay in dealing with this problem. One of our principal machines here decided to die in a permanent fashion, and we have had to spend some time scurrying around picking up the pieces. Mark Davis: Yes, please remove the EXPOTECH address from your subsidiary list. As long as APPLE.COM is on the Internet, there is no need for the mail to clog your system by hopping in and out again. Let us know when you have removed the address, and then we'll add it here. Catherine-Aimee and Gwynnyd: My ladies, as you can discern from the messages above, we don't know how your address is given on the mailing list, because your address isn't directly on the main list here. When Mr. Davis or Mistress Siobhan has removed you at UMichigan, we will be glad to add you here in the form you suggest. That may not, however, solve the problem, as I explain below. AppleLink Postmaster: This is the bounce message we are receiving. Someone in your part of the world seems to think that any mail over 32K long is too long. That's very restrictive, and in the case of digested mailing lists really can't be dealt with by having the author split the mail into smaller chunks. Such lists often have digests longer than 32K -- our automatic digestifying program is tuned to keep digests under 50K, which seems to be the trip point for many systems. And administrators of such lists simply don't have the time to specially cater to a small number of list members whose systems complain in this fashion. Anything you can do to help would be appreciated. Thank you. Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU; 22 Apr 90 03:58:30 EDT Received: from umich.edu by mintaka.lcs.mit.edu id aa18158; 22 Apr 90 3:53 EDT Received: from ummts.cc.umich.edu by umich.edu (5.61/1123-1.0) id AA13505; Sun, 22 Apr 90 03:52:58 -0400 Received: from umix.cc.umich.edu by um.cc.umich.edu via Internet with TCP; Sun, 22 Apr 90 03:52:40 EDT Received: by umix.cc.umich.edu id AA05734; Sun, 22 Apr 90 03:42:35 EDT Received: from [90.1.0.10] by apple.com with SMTP id AA06981; Sun, 22 Apr 90 00:38:45 -0700 for @um.cc.umich.edu:SCA-REQUEST@mc.lcs.mit.edu Received: from apple.com by goofy.apple.com with SMTP id AA22231; Sun, 22 Apr 90 00:38:22 -0700 for fair@apple.com Date: Sun, 22 Apr 90 00:38:22 -0700 From: Mail Delivery Subsystem Message-Id: <9004220738.AA22231@internal.apple.com> To: bounces@apple.com To: @um.cc.umich.edu:SCA-REQUEST@mc.lcs.mit.edu MMDF-Warning: Parse error in original version of preceding line at lcs.mit.edu Subject: Returned mail: Service unavailable ----- Transcript of session follows ----- The maximum amount of text that a Macintosh "text edit" window can display is 32K. Please break up your correspondence into pieces that are smaller than that, and resend. letter too large (39461 bytes) from @Mintaka.lcs.mit.edu:SCA-Request@MC.LCS.MIT.EDU to expotech 554 ... Service unavailable ----- Unsent message follows ----- Received: from apple.com by goofy.apple.com with SMTP (5.61/25-eef) id AA22229; Sun, 22 Apr 90 00:38:22 -0700 for expotech Received: from umich.edu by apple.com with SMTP (5.61/25-eef) id AA06974; Sun, 22 Apr 90 00:38:15 -0700 for EXPOTECH@applelink.apple.com Received: from ummts.cc.umich.edu by umich.edu (5.61/1123-1.0) id AA13289; Sun, 22 Apr 90 03:38:04 -0400 Received: from umix.cc.umich.edu by um.cc.umich.edu via Internet with TCP; Sun, 22 Apr 90 03:37:47 EDT Received: by umix.cc.umich.edu id AA05336; Sun, 22 Apr 90 03:31:20 EDT Received: from mc.lcs.mit.edu by mintaka.lcs.mit.edu id aa17020; 22 Apr 90 3:22 EDT Date: 22 Apr 90 03:11:59 EDT From: Automatic SCA Digestifier <@Mintaka.lcs.mit.edu:SCA-Request@MC.LCS.MIT.EDU> To: SCA%MC.LCS.MIT.EDU@Mintaka.lcs.mit.edu Subject: SCA Digest #671 [digest text omitted] Lady Eowyn Eilonwy of Alewife Brook, Deputy List Bureaucrat  Date: Mon, 7 May 90 16:15:44 EDT From: "Pandora B. Berman" To: MLY%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU cc: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <665901.900507.CENT@MC.LCS.MIT.EDU> Date: Fri, 27 Apr 90 02:54:05 EDT From: Alan Bawden Subject: Digestification software To: MLY%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU cc: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Date: Thu, 26 Apr 90 11:42:00 EDT From: Richard Mlynarik Subject: Digestification software To: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU cc: MLY%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Using the advanced ITS  feature I hear that there is some consideration of using some mess of unix shell scripts to try to support the present crop of digests in the brave new future. I think a much better idea would be for me to volunteer to write the necessary code in gnu emacs (the only decent system programming environment for unix!) Sounds like an excellent idea to me. I forget if you grok MIDAS code, but if you do, you should look through MC:DIGEST;DIGEST > to see what the current one does. I'm pretty happy with it. i don't mean to rush you, but what's the prognosis on this? alan would like to start shutting things down in about two weeks. i have received copies of two things which claim to be unixish digestifiers, which i haven't yet tried to show to anyone knowledgeable about unix, so i have no idea how close they would come to our current mode of operations or how effective they would be.  Date: Fri, 27 Apr 90 02:54:05 EDT From: Alan Bawden Subject: Digestification software To: MLY%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU cc: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Reply-to: Alan@Reagan.AI.MIT.EDU In-reply-to: Msg of Thu 26 Apr 90 11:42:00 EDT from Richard Mlynarik Message-ID: <663499.900427.ALAN@MC.LCS.MIT.EDU> Date: Thu, 26 Apr 90 11:42:00 EDT From: Richard Mlynarik ... I think a much better idea would be for me to volunteer to write the necessary code in gnu emacs ... If anybody wants me to act on this offer, speak up. Sounds like an excellent idea to me. I forget if you grok MIDAS code, but if you do, you should look through MC:DIGEST;DIGEST > to see what the current one does. I'm pretty happy with it.  Date: Thu, 26 Apr 90 11:42:00 EDT From: Richard Mlynarik Subject: Digestification software To: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU cc: MLY%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <663307.900426.MLY@MC.LCS.MIT.EDU> Using the advanced ITS  feature I hear that there is some consideration of using some mess of unix shell scripts to try to support the present crop of digests in the brave new future. I think a much better idea would be for me to volunteer to write the necessary code in gnu emacs (the only decent system programming environment for unix!) If anybody wants me to act on this offer, speak up.  Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU; 20 Apr 90 11:48:45 EDT Received: from A.GP.CS.CMU.EDU by mintaka.lcs.mit.edu id aa22983; 20 Apr 90 11:46 EDT From: Christopher Maeda Date: Fri, 20 Apr 90 11:45:14 EDT To: mt@media-lab.media.mit.edu CC: bug-digest@MC.lcs.mit.edu In-reply-to: Michael Travers's message of Fri, 20 Apr 90 00:29 EDT <19900420042942.4.MT@OUROBOROS.MEDIA.MIT.EDU> Subject: Indigestion Reply-to: cmaeda@cs.cmu.edu Message-ID: <9004201146.aa22983@mintaka.lcs.mit.edu> Since aviation has become entirely a usenet forward, I was thinking of writing something that will grab news from an nntp server and digest it. Anyone know if something like that has been done already?  Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU; 20 Apr 90 09:55:03 EDT Received: from life.ai.mit.edu by mintaka.lcs.mit.edu id aa17297; 20 Apr 90 9:51 EDT Received: from PIGPEN.AI.MIT.EDU by life.ai.mit.edu (4.1/AI-4.10) id AA27180; Fri, 20 Apr 90 09:50:24 EDT Date: Fri, 20 Apr 90 09:47 EDT From: Alan Bawden Subject: Indigestion To: cygint!Gumby@labrea.stanford.edu, mt@media-lab.media.mit.edu Cc: bug-digest@mc.lcs.mit.edu In-Reply-To: <9004200532.AA18951@cygnus.com> Message-Id: <19900420134759.6.ALAN@PIGPEN.AI.MIT.EDU> Date: Thu, 19 Apr 90 22:32:37 PDT From: Gumby Vinayak Wallace ... Contact the guy who now maintains dead-flames. His name's Mark Rouleau; I think his address is mer6@virginia.edu. I dunno; his phone numbers are (were?) 804 924 0631 (W) or 804 971 7473 (H). Hmm, I think I left his name in the NAMES file on MC in a comment near the entry for dead-flames-digest. I reached him once by mailing to Dead-Flames-Request@MC (which forwards to Dead-Flames-Request@Virginia.EDU).  Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU; 20 Apr 90 01:52:55 EDT Received: from labrea.Stanford.EDU by mintaka.lcs.mit.edu id aa01005; 20 Apr 90 1:50 EDT Received: by labrea.stanford.edu; Thu, 19 Apr 90 21:50:47 PST Received: by cygnus.com (4.0/SMI-4.0) id AA18951; Thu, 19 Apr 90 22:32:37 PDT Date: Thu, 19 Apr 90 22:32:37 PDT From: Gumby Vinayak Wallace Message-Id: <9004200532.AA18951@cygnus.com> To: mt@media-lab.media.mit.edu Cc: bug-digest@mc.lcs.mit.edu Reply-To: cygint!Gumby@labrea.stanford.edu In-Reply-To: Michael Travers's message of Fri, 20 Apr 90 00:29 EDT <19900420042942.4.MT@OUROBOROS.MEDIA.MIT.EDU> Subject: Indigestion Date: Fri, 20 Apr 90 00:29 EDT From: Michael Travers So, does anybody know of any decent digesters that run on something besides ITS? I have a package called Digest.Kit that comes from HP and consists of horrible unix shell scripts and such, but I don't think I will like it. Contact the guy who now maintains dead-flames. His name's Mark Rouleau; I think his address is mer6@virginia.edu. I dunno; his phone numbers are (were?) 804 924 0631 (W) or 804 971 7473 (H). Hmm, I think I left his name in the NAMES file on MC in a comment near the entry for dead-flames-digest.  Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU; 20 Apr 90 00:38:54 EDT Received: from media-lab.media.mit.edu by mintaka.lcs.mit.edu id aa28552; 20 Apr 90 0:34 EDT Received: by media-lab (5.57/4.8) id AA05909; Fri, 20 Apr 90 00:34:31 EDT Date: Fri, 20 Apr 90 00:29 EDT From: Michael Travers Subject: Indigestion To: bug-digest@MC.lcs.mit.edu Message-Id: <19900420042942.4.MT@OUROBOROS.MEDIA.MIT.EDU> So, does anybody know of any decent digesters that run on something besides ITS? I have a package called Digest.Kit that comes from HP and consists of horrible unix shell scripts and such, but I don't think I will like it.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU; 15 Jan 90 02:54:28 EST Date: Mon, 15 Jan 90 02:55:55 EST From: "Pandora B. Berman" Subject: feature wanted To: BUG-DIGEST%AI.AI.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <688844.900115.CENT@AI.AI.MIT.EDU> Date: 15 Jan 90 02:17:08 EST From: Automatic SCA Digestifier To: SCA%MC.LCS.MIT.EDU@Mintaka.LCS.MIT.EDU Subject: SCA Digest #573 Today's Topics: Arval's blazon Events near Dayton there were two topics for today's digest (see above), but three included msgs -- two msgs had the same topic. please implement the proposal to include a count with topics used multiple times. the idea of adding, below the topics, a line to the effect of "there were 17 msgs which gave no subject" is also a good idea..  Date: Sat, 13 Jan 90 04:22:38 EST From: Alan Bawden Subject: OK, so now we have -three- digestifiers running on MC... To: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <637839.900113.ALAN@MC.LCS.MIT.EDU> I have written a new digestifier. Currently only the SCA digest is using it, but I would like to convert the others as soon as I am satisfied that it doesn't have any problems. See MC:DIGEST;DIGEST ORDER for documentation, MC:DIGEST;DEFS > for individual digest definitions, and MC:DIGEST;LOG > for a log of its actions.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 5 Sep 89 14:48:09 EDT Date: Tue, 5 Sep 89 14:50:35 EDT From: David Chapman Subject: Digestifier To: "bawden@PARC.XEROX.COM"@MINTAKA.LCS.MIT.EDU cc: BUG-DIGEST@MC.LCS.MIT.EDU, Postmaster@MC.LCS.MIT.EDU In-reply-to: Msg of Tue 5 Sep 89 10:46 PDT from Alan Bawden Message-ID: <640987.890905.ZVONA@AI.AI.MIT.EDU> Perhaps somebody should fix the digestifier to put a legal address in the header. There is no machine named "MINTAKA.MC.LCS.EDU". I though only Unix weenies made mistakes like that, not people who dare to touch PDP-10 code. My fault. What an amazing braino. I'm quite surprised no one has noticed it before. (Maybe many people did and sent complaints to foo-request%mc.lcs.mit.edu@mintaka.mc.lcs.edu?) Anyway. Fixed in the source and installed.  Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU 5 Sep 89 13:54:07 EDT Received: from arisia.Xerox.COM by mintaka.lcs.mit.edu id aa07683; 5 Sep 89 13:49 EDT Received: from roo.parc.Xerox.COM by arisia.Xerox.COM with SMTP (5.61+/IDA-1.2.8/gandalf) id AA15855; Tue, 5 Sep 89 10:45:05 -0700 Received: from mr-bun.parc.Xerox.COM by roo.parc.xerox.com with SMTP (5.61+/IDA-1.2.8/gandalf) id AA02295; Tue, 5 Sep 89 10:48:06 PDT Date: Tue, 5 Sep 89 10:46 PDT From: Alan Bawden Subject: Digestifier To: Postmaster@mc.lcs.mit.edu Cc: BUG-DIGEST@mc.lcs.mit.edu Message-Id: <19890905174629.9.ALAN@MR-BUN.parc.xerox.com> Perhaps somebody should fix the digestifier to put a legal address in the header. There is no machine named "MINTAKA.MC.LCS.EDU". I though only Unix weenies made mistakes like that, not people who dare to touch PDP-10 code. Date: 4 SEP 89 00:04:44 EDT From: Automatic SubGenius Digestifier Subject: SubGenius Digest #260 To: SubGenius%mc.lcs.mit.edu@mintaka.mc.lcs.edu Reply-To: SubGenius%mc.lcs.mit.edu@mintaka.mc.lcs.edu Message-Id: <8909040029.aa11726@mintaka.lcs.mit.edu> SubGenius Digest #260 4 SEP 89 00:04:44 EDT I suppose I should also complain about the Scheme digest that showed up truncated the other day. I don't know where it got truncated (and I'm actually glad it was, because some pinhead tried to mail out the entire XScheme sources), but some IO screwup in the digestifier would be a good candidate.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 1 Aug 89 21:41:06 EDT Date: Tue, 1 Aug 89 21:41:00 EDT From: David Chapman To: BUG-DIGEST%AI.AI.MIT.EDU@MINTAKA.LCS.MIT.EDU In-reply-to: Msg of Sat 29 Jul 89 11:58:51 EDT from David Chapman Message-ID: <628385.890801.ZVONA@AI.AI.MIT.EDU> Date: Sat, 29 Jul 89 11:58:51 EDT From: David Chapman To: BUG-DIGEST%MC.LCS.MIT.EDU at MINTAKA.LCS.MIT.EDU The digester ought to make %'ed reply-to: fields, since a lot of people's mailers don't grok MX records and can't reply to MC. I'll fix this in a crockish way soon if someone doesn't do it in a clean way... (I don't know MIDAS, but I do know FORMAT!) I did this.  Date: Sat, 29 Jul 89 11:58:51 EDT From: David Chapman To: BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <602793.890729.ZVONA@MC.LCS.MIT.EDU> The digester ought to make %'ed reply-to: fields, since a lot of people's mailers don't grok MX records and can't reply to MC. I'll fix this in a crockish way soon if someone doesn't do it in a clean way... (I don't know MIDAS, but I do know FORMAT!)  Received: from lcs.mit.edu (CHAOS 15044) by MC.LCS.MIT.EDU 26 May 89 14:24:38 EDT Received: from reagan.ai.mit.edu by mintaka.lcs.mit.edu id aa23726; 26 May 89 14:21 EDT Received: from PIGPEN.AI.MIT.EDU by REAGAN.AI.MIT.EDU via CHAOS with CHAOS-MAIL id 214311; Fri 26-May-89 14:20:25 EDT Date: Fri, 26 May 89 14:20 EDT From: Alan Bawden Subject: fryer To: MAEDA%MC.LCS.MIT.EDU@lcs.mit.edu cc: SRA%MC.LCS.MIT.EDU@lcs.mit.edu, BUG-DIGEST%MC.LCS.MIT.EDU@lcs.mit.edu In-Reply-To: <584376.890526.MAEDA@MC.LCS.MIT.EDU> Message-ID: <19890526182020.7.ALAN@PIGPEN.AI.MIT.EDU> Date: Fri, 26 May 89 06:15:42 EDT From: "Christopher M. Maeda" ... If it doesn't lose, I'll put in the rest in a couple of days. Please wait until I look at it before giving it to the other digests.  Date: Fri, 26 May 89 06:15:42 EDT From: "Christopher M. Maeda" Subject: fryer To: alan@AI.AI.MIT.EDU cc: SRA%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU, BUG-DIGEST%MC.LCS.MIT.EDU@MINTAKA.LCS.MIT.EDU Message-ID: <584376.890526.MAEDA@MC.LCS.MIT.EDU> I finished the error handlers for Fryer and tested it out on some digests. It seems to work so I installed it on MC. It's only set up to do the aviation digest. If it doesn't lose, I'll put in the rest in a couple of days. The code is in mc:maeda;fryer >. The aviation digest is handled by dragon;daily digest.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 31 Mar 89 03:17:21 EST Date: Fri, 31 Mar 89 03:17:52 EST From: "Christopher M. Maeda" To: CENT@AI.AI.MIT.EDU cc: ALAN@AI.AI.MIT.EDU, SRA@LCS.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU In-reply-to: Msg of Fri 31 Mar 89 02:33:11 EST from Pandora B. Berman Message-ID: <567012.890331.MAEDA@AI.AI.MIT.EDU> Date: Fri, 31 Mar 1989 02:10 EST From: Rob Austein To: "Pandora B. Berman" , MAEDA@AI.AI.MIT.EDU Cc: Bug-DIGEST@MC.LCS.MIT.EDU FYI, -all- the digests are running the new version of the digestifier. Chris has been overexposed to unix and I guess he thought he was making a hard link, not a symbolic link. well, fuck me harder. alan says he thought he urged chris to de-install the new one because he (alan) had discovered by eyeballing the code that it wouldn't work. if chris didn't, please do it yourself. Huh? The new digest is dragon;dgstbg bin. I did a ":link daily aviatn,dgstbg bin" and then deleted it the next day and linked it back to digest bin. I don't think any of the digests were running the new software (except aviatn for one day). I got another "digest too big" bounce after I put back aviatn, too. I could be wrong though. What *is* the difference between a hard link and a symbolic link?  Received: from XX.LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 31 Mar 89 03:12:09 EST Date: Fri, 31 Mar 1989 03:13 EST Message-ID: From: Rob Austein To: "Pandora B. Berman" Cc: ALAN@AI.AI.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU, MAEDA@AI.AI.MIT.EDU, sra@XX.LCS.MIT.EDU In-reply-to: Msg of 31 Mar 1989 02:33-EST from "Pandora B. Berman" I jumped to confusions. Should have said "all the digests are running from the same binary" (ie, all of Puff's links point at the same file). Closer examination shows that I mis-read the (stupid numeric month format) date and that the version that's actually running is one I compiled on 24 Feb 89. Sorry for the false alarm.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 31 Mar 89 02:32:46 EST Date: Fri, 31 Mar 89 02:33:11 EST From: "Pandora B. Berman" To: SRA@LCS.MIT.EDU cc: CENT@AI.AI.MIT.EDU, MAEDA@AI.AI.MIT.EDU, ALAN@AI.AI.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU Message-ID: <566995.890331.CENT@AI.AI.MIT.EDU> Date: Fri, 31 Mar 1989 02:10 EST From: Rob Austein To: "Pandora B. Berman" , MAEDA@AI.AI.MIT.EDU Cc: Bug-DIGEST@MC.LCS.MIT.EDU FYI, -all- the digests are running the new version of the digestifier. Chris has been overexposed to unix and I guess he thought he was making a hard link, not a symbolic link. well, fuck me harder. alan says he thought he urged chris to de-install the new one because he (alan) had discovered by eyeballing the code that it wouldn't work. if chris didn't, please do it yourself..  Received: from XX.LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 31 Mar 89 02:09:01 EST Date: Fri, 31 Mar 1989 02:10 EST Message-ID: From: Rob Austein To: "Pandora B. Berman" , MAEDA@AI.AI.MIT.EDU Cc: Bug-DIGEST@MC.LCS.MIT.EDU In-reply-to: Msg of 30 Mar 1989 23:30-EST from "Pandora B. Berman" FYI, -all- the digests are running the new version of the digestifier. Chris has been overexposed to unix and I guess he thought he was making a hard link, not a symbolic link.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 26 Mar 89 08:29:31 EST Received: from JONES.AI.MIT.EDU by AI.AI.MIT.EDU via Chaosnet; 26 MAR 89 08:29:22 EST Date: Sun, 26 Mar 89 08:27 EST From: Christopher Maeda Subject: digestifier To: bug-digest@MC.LCS.MIT.EDU, bug-mail@MC.LCS.MIT.EDU Message-ID: <19890326132732.1.MAEDA@JONES.AI.MIT.EDU> I just fixed it to break up large digests. It counts characters and messages while parsing the subject lines. Once it reads more than the maximum number of characters, it quits reading in messages and writes them out instead. Any unread messages are then copied to a file called JNAME _OLD_. The digestifier then writes another digest. It stops when there is no more _OLD_ files and no inbox left. These changes are in the latest version of digest > in mc:gumby; I copied the binary to dragon;dgstbg bin and linked the daily aviatn demon to it. If everything goes ok for the next couple of days, we can link the rest of the digests to it. Chris  Date: Fri, 24 Feb 89 23:36:03 EST From: Rob Austein Subject: Digests moved to COMSAT BULK To: DEAD-FLAMES-REQUEST@MC.LCS.MIT.EDU, AVIATION-REQUEST@MC.LCS.MIT.EDU, PAGANISM-REQUEST@MC.LCS.MIT.EDU, SCA-REQUEST@MC.LCS.MIT.EDU, SUBGENIUS-REQUEST@MC.LCS.MIT.EDU, SCHEME-REQUEST@MC.LCS.MIT.EDU cc: BUG-MAIL@MC.LCS.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU Reply-To: Bug-MAIL@AI.AI.MIT.EDU Message-ID: <544373.890224.SRA@MC.LCS.MIT.EDU> Now that we have all of Alan's good work on making COMSAT use the LOCK: device, it seemed ok to move the digests to COMSAT BULK. So I did. The actual switch was a one line change to GUMBY;DIGEST >. The only change this entails from the point of view of a digest administrator is that you should look in MC:.BULK.; for [O]STATS files if you care about such things, rather than in MC:.MAIL.;. The various mailing lists still reside in MC:.MAIL.;NAMES > and should remain there; the NAMES file in .BULK. should never be anything besides a link to the one in .MAIL.; or chaos will ensue. Obviously, any digests that have already been sent out via regular COMSAT will remain in regular COMSAT's queue until they are delivered or time out. But digests posted from now on will go via COMSAT BULK. The overall result of this change should be improved performance for both digests and regular mail, since the main effect will be to lessen contention for MC's regular COMSAT's limited processing time. Please report any problems you think may have been caused by this change to BUG-MAIL. --Rob  Date: Fri, 24 Feb 89 05:27:18 EST From: Alan Bawden Subject: Local mailbox screw eliminated To: BUG-MAIL@MC.LCS.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU, NICK@MC.LCS.MIT.EDU Message-ID: <1915.890224.ALAN@MC.LCS.MIT.EDU> COMSAT now uses the LOCK: device to coordinate access to local mailboxes. See the code for gory details. MC:.BULK.;NAMES 999999 is now a link to MC:.MAIL.;NAMES >. In theory, new mail can be placed in either .MAIL.;MAIL > -or- .BULK.;MAIL > and it should be delivered to exactly the same places. Mail hacker visible changes: 1. The newest version of COMSAT will not run under an ITS without the LOCK: device. Under ITS versions before 1630 COMSAT will crash. Currently AI, MC, and ML are running ITS version 1632, and the new COMSAT is installed on all of them. If we have to revert to an earlier version of ITS on any of these machines (heaven forbid!) then we will have to revert to the previous COMSAT, and on MC we would have to shut down the BULK COMSAT completely (if you don't see why, you haven't thought about it hard enough!). 2. Occasionally you will now see a line in the STATS file like: 030310 TO->GLURK(USERS1;)... Inbox locked. This means that mailer found that some other COMSAT (or the GMSGS program, which I hacked to do this locking as well) was in the process of hacking the mailbox in question. This will be treated as a temporary error, and the mail will be queued for later delivery. 3. Since -both- mailers on MC compile the NAMES file separately, it now takes -double- the CPU time to install a new NAMES file. This is a waste since both compilations (presumably) result in equivalent LIST EQV files. Sorry. 4. I copied the entries for the AIList mailing list from the .BULK.;NAMES > into .MAIL.;NAMES > (these were the only BULK-specific mailing lists). No one should -ever- create a separate NAMES file for the BULK COMSAT again. 5. Since both mailers have the same NAMES file, they now both deliver failed mail to .MAIL.;FAILED STUFF. Please note that the BULK COMSAT continues to be a totally independent mailer. It shares the NAMES file, but all of its queues and databases are its own. I suppose the various digests can be switched over to using .BULK. whenever their maintainers feel that they want to give it a try. (The LOCK: device is documented in .INFO.;ITS LOCKS for those that want to learn more about how this works.) (To help test this all out, this mail will be delivered by the BULK COMSAT.)  Received: from REAGAN.AI.MIT.EDU (CHAOS 13065) by MC.LCS.MIT.EDU 23 Feb 89 21:11:17 EST Received: from QUESTION-AUTHORITY.AI.MIT.EDU by REAGAN.AI.MIT.EDU via CHAOS with CHAOS-MAIL id 176157; Thu 23-Feb-89 21:11:44 EST Date: Thu, 23 Feb 89 21:08 EST From: Alan Bawden Subject: Which mailer should digests use? To: GUMBY@AI.AI.MIT.EDU cc: CENT@AI.AI.MIT.EDU, SRA@XX.LCS.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU In-Reply-To: <543211.890223.GUMBY@AI.AI.MIT.EDU> Message-ID: <19890224020836.2.ALAN@QUESTION-AUTHORITY.AI.MIT.EDU> Date: Thu, 23 Feb 89 02:52:52 EST From: David Vinayak Wallace Date: Wed, 22 Feb 89 02:32:36 EST From: Pandora B. Berman then there would be the possibility of getting both comsats to use a single NAMES file. Yow, what an idea. Why not; I guess they both use the same lsr file. Bulk shouldn't write FAILED files; if it lost presumably so would the mailer in .mail. I presume by "FAILED files" you are talking about the files named "NAMED ERRnnn"? Since the bulk Comsat would be writing them in .BULK. I don't see why it should be prevented from generating them. alan also just wondered what would happen if both comsats tried to put stuff into FAILED STUFF; how will the loser handle the resulting error? I don't understand; couldn't it just block? This seems to me to be the same as someone typing gumby and then going to lunch while it's in a more break. When COMSAT discovers that someone typed gumby and so it gets a FILE LOCKED error it doesn't "block". Insead it gets a temporary error, fails to deliver the message, and requeues it. Comsat never blocks delivering mail, nor should it when we introduce the appropriate locking. When a Comsat finds that a local mailbox is locked by another Comsat, it will treat the error as a temporary error and requeue the message for later. I was wondering about FAILED STUFF because I couldn't recall whether it was a special case, being the destination of last resort, or just another mailing list. If it really is just another mailing list, then there shouldn't be any problem.  Received: from LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 23 Feb 89 20:52:55 EST Date: Thu, 23 Feb 1989 20:52 EST Message-ID: From: Rob Austein To: David Chapman Cc: BUG-digest@MC.LCS.MIT.EDU, BUG-mail@MC.LCS.MIT.EDU, Alan@AI.AI.MIT.EDU Subject: I >thought< I remembered there being a more complex screw: Date: Thursday, 23 February 1989 20:02-EST From: David Chapman Date: Wed, 26 Oct 88 17:25 EDT From: Alan Bawden Hold on. It's not as easy as you think to control local delivery. Recall that people can change their INQUIR entries to say that mail should be sent to them @MC, which the BULK mailer will see, and then they can edit .MAIL.;NAMES > to say where it should really go, which the BULK mailer will -not- see, and it will thus start delivering locally. Also, when the host table changes, and an address suddenly becomes unparsable, the mailer will revert to local delivery. Huh? If COMSAT BULK sends mail to USER@AI and USER has her INQUIR entry forwarding mail to USER@MC, AI will ship the mail back to MC's normal COMSAT, which will see .MAIL.;NAMES and will do the right thing. That's the whole point of sending the mail to AI instead of having COMSAT BULK deliver it locally on MC. If AI becomes an unparsable address I think we will have bigger things to worry about than where the subgenius mailing list goes.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 23 Feb 89 20:01:39 EST Date: Thu, 23 Feb 89 20:02:09 EST From: David Chapman Subject: I >thought< I remembered there being a more complex screw: To: BUG-digest@MC.LCS.MIT.EDU, BUG-mail@MC.LCS.MIT.EDU Message-ID: <543767.890223.ZVONA@AI.AI.MIT.EDU> Date: Wed, 26 Oct 88 17:25 EDT From: Alan Bawden Subject: bulk digests To: ZVONA@AI.AI.MIT.EDU cc: BUG-MAIL@AI.AI.MIT.EDU Date: Tue, 25 Oct 88 14:59:15 EDT From: David Chapman So as long as there are no local recipients on a BULK list, it's OK? That's easy, then: indirect local recpients, if any, through AI or someplace. Should we start systematically moving big lists to BULK? Hold on. It's not as easy as you think to control local delivery. Recall that people can change their INQUIR entries to say that mail should be sent to them @MC, which the BULK mailer will see, and then they can edit .MAIL.;NAMES > to say where it should really go, which the BULK mailer will -not- see, and it will thus start delivering locally. Also, when the host table changes, and an address suddenly becomes unparsable, the mailer will revert to local delivery. It would be better if we were to first introduce a lock to control local delivery, and then arrange for the BULK mailer to share the NAMES file somehow.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 23 Feb 89 15:51:14 EST Date: Thu, 23 Feb 89 15:51:42 EST From: David Chapman To: BUG-digest@MC.LCS.MIT.EDU, postmaster@MC.LCS.MIT.EDU Message-ID: <543586.890223.ZVONA@AI.AI.MIT.EDU> While I think of it: there's one thing that running the digests through bulk WON'T do for us, which is probably the thing we actually want, and that is to improve percieved mailer response. The digests go out at night, when not many people are on. During the day they contribute very little to mailer load. The SIMTEL stuff comes in during the day (at random, essentially, whenever SIMTEL thinks to talk to MC and (for some bizzare reason) its actually willing to accept mail). I get a fair amount of my mail through MC and am frustrated at the slow turn around; bulking the digests won't help with that.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 23 Feb 89 08:07:56 EST Received: from NICO.AI.MIT.EDU by AI.AI.MIT.EDU via Chaosnet; 22 FEB 89 15:16:28 EST Date: Wed, 22 Feb 89 15:13 EST From: Christopher Maeda Subject: Bulk Mailer Screw Conditions To: SRA@XX.LCS.MIT.EDU cc: bug-digest@MC.LCS.MIT.EDU, jtw@XX.LCS.MIT.EDU, alan@AI.AI.MIT.EDU In-Reply-To: Message-ID: <19890222201349.3.MAEDA@NICO.AI.MIT.EDU> Date: Mon, 20 Feb 1989 23:01 EST From: Rob Austein Date: Monday, 20 February 1989 17:23-EST From: David Chapman But you didn't answer my questions: what are the screw conditions, and how do we do this? We do it by changing the word ".MAIL." to the word ".BULK." in the digestifier source code. This sounds so simple even I could do it. (I even volunteer.) What are the differences between the two comsats? Do they really not require any of the headers to be munged in the generated message (or anything else like that)? The screws that I know of involve the lack of any file locking protocol that will keep the two COMSATs from treading on each other's toes should they happen to attempt to update the same file at the same time. At one point Alan had some plan for fixing this but I don't know if it got finished. The usual fix at the moment is to use separate inboxes for the two mailers. Eg, ARPANET-BBOARDS-BUGS (the SMTP source path given in ANBB postings) traslates to two different filenames: .MAIL.;NAMES translates it to ANBB;BBOARD BUGS and .BULK.;NAMES translates it to ANBB;BBOARD ANTS. Not a big deal. Delivering to a local (on MC) user mailbox via COMSAT BULK is a bad idea, for similar reasons; fortunately, don't have any user mailboxes on MC, so we don't care. Does this screw case just apply to MC since there are two comsats running? I can see how local delivery would cause the comsats to step on each other. I don't see how this would apply otherwise. There is also a screw involving the way some people forard their mail using both NAMES > and INQUIR, which Alan can describe if you really care; the short answer is that mail sent through COMSAT BULK to somebody whose INQUIR entry says that they read their mail on MC should be sent to some other ITS machine (probably AI) instead of attempting to deliver it locally. I modified aviation's distribution file so that any local (mc) addresses get sent to ai. Is this the simple solution?  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 23 Feb 89 08:07:46 EST Date: Thu, 23 Feb 89 02:52:52 EST From: David Vinayak Wallace Subject: Which mailer should digests use? To: CENT@AI.AI.MIT.EDU cc: ALAN@AI.AI.MIT.EDU, SRA@LCS.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU In-reply-to: Msg of Wed 22 Feb 89 02:32:36 EST from Pandora B. Berman Message-ID: <543211.890223.GUMBY@AI.AI.MIT.EDU> Date: Wed, 22 Feb 89 02:32:36 EST From: Pandora B. Berman then there would be the possibility of getting both comsats to use a single NAMES file. Yow, what an idea. Why not; I guess they both use the same lsr file. Bulk shouldn't write FAILED files; if it lost presumably so would the mailer in .mail. alan also just wondered what would happen if both comsats tried to put stuff into FAILED STUFF; how will the loser handle the resulting error? I don't understand; couldn't it just block? This seems to me to be the same as someone typing gumby and then going to lunch while it's in a more break.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 23 Feb 89 08:07:41 EST Date: Thu, 23 Feb 89 02:47:34 EST From: David Vinayak Wallace Subject: Which mailer should digests use? To: SRA@MC.LCS.MIT.EDU cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU In-reply-to: Msg of Sat 18 Feb 89 05:16:24 EST from Rob Austein Message-ID: <543209.890223.GUMBY@AI.AI.MIT.EDU> This is a good idea if you can get the comsats to lock each other out. I think the restrictions on local addresses probably aren't too bad (just tell digest maintainers that each name on their list should be qualified -- and local addresses should appear as foo@ai).  Received: from LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 23 Feb 89 00:18:34 EST Date: Wed, 22 Feb 1989 16:10 EST Message-ID: From: Rob Austein To: Christopher Maeda Cc: alan@AI.AI.MIT.EDU, bug-digest@MC.LCS.MIT.EDU, jtw@LCS.MIT.EDU, sra@LCS.MIT.EDU Subject: Bulk Mailer Screw Conditions In-reply-to: Msg of 22 Feb 1989 15:13-EST from Christopher Maeda Date: Wednesday, 22 February 1989 15:13-EST From: Christopher Maeda We do it by changing the word ".MAIL." to the word ".BULK." in the digestifier source code. This sounds so simple even I could do it. (I even volunteer.) What are the differences between the two comsats? Do they really not require any of the headers to be munged in the generated message (or anything else like that)? I don't see why it would require special munging, it's just another COMSAT. You do have to think about what you're doing, but you have to do that with the current setup too. Does this screw case just apply to MC since there are two comsats running? Right. I modified aviation's distribution file so that any local (mc) addresses get sent to ai. Is this the simple solution? That should handle the INQUIR problem. The worse case if you do that is that AI will ship it back to MC's regular COMSAT for delivery.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 22 Feb 89 03:39:39 EST Date: Wed, 22 Feb 89 02:32:36 EST From: "Pandora B. Berman" Subject: Which mailer should digests use? To: SRA@LCS.MIT.EDU cc: ALAN@AI.AI.MIT.EDU, BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU Message-ID: <542452.890222.CENT@AI.AI.MIT.EDU> Date: Mon, 20 Feb 1989 23:01 EST From: Rob Austein To: David Chapman Cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU Subject: Which mailer should digests use? .... The screws that I know of involve the lack of any file locking protocol that will keep the two COMSATs from treading on each other's toes should they happen to attempt to update the same file at the same time. At one point Alan had some plan for fixing this but I don't know if it got finished.... alan has not been partaking of this discussion because he isn't on postmaster or bug-digest (and says he doesn't want to be). i have forwarded to him the relevant pieces of this discussion. he says the fix to keep the 2 comsats from tromping on each other is to insert a lock into the place in comsat that does local delivery. if someone will kindly inform him where to find this, he can insert the lock. then there would be the possibility of getting both comsats to use a single NAMES file. alan also just wondered what would happen if both comsats tried to put stuff into FAILED STUFF; how will the loser handle the resulting error?  Received: from LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 20 Feb 89 23:03:04 EST Date: Mon, 20 Feb 1989 23:01 EST Message-ID: From: Rob Austein To: David Chapman Cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU Subject: Which mailer should digests use? In-reply-to: Msg of 20 Feb 1989 17:23-EST from David Chapman Date: Monday, 20 February 1989 17:23-EST From: David Chapman But you didn't answer my questions: what are the screw conditions, and how do we do this? We do it by changing the word ".MAIL." to the word ".BULK." in the digestifier source code. The screws that I know of involve the lack of any file locking protocol that will keep the two COMSATs from treading on each other's toes should they happen to attempt to update the same file at the same time. At one point Alan had some plan for fixing this but I don't know if it got finished. The usual fix at the moment is to use separate inboxes for the two mailers. Eg, ARPANET-BBOARDS-BUGS (the SMTP source path given in ANBB postings) traslates to two different filenames: .MAIL.;NAMES translates it to ANBB;BBOARD BUGS and .BULK.;NAMES translates it to ANBB;BBOARD ANTS. Not a big deal. Delivering to a local (on MC) user mailbox via COMSAT BULK is a bad idea, for similar reasons; fortunately, don't have any user mailboxes on MC, so we don't care. There is also a screw involving the way some people forard their mail using both NAMES > and INQUIR, which Alan can describe if you really care; the short answer is that mail sent through COMSAT BULK to somebody whose INQUIR entry says that they read their mail on MC should be sent to some other ITS machine (probably AI) instead of attempting to deliver it locally.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 20 Feb 89 17:23:46 EST Date: Mon, 20 Feb 89 17:23:43 EST From: David Chapman Subject: Which mailer should digests use? To: SRA@LCS.MIT.EDU cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU In-reply-to: Msg of Mon 20 Feb 1989 17:02 EST from Rob Austein Message-ID: <541433.890220.ZVONA@AI.AI.MIT.EDU> The digest stuff, on the other hand, is simply a silly use of the resources that we already have. COMSAT BULK is nearly idle, and the digests are the one thing that could easily be moved there. Well, as I said, I think that bulkifying digsts is a reasonable thing to do, regardless of the resolution of the SIMTEL issue. But you didn't answer my questions: what are the screw conditions, and how do we do this?  Received: from LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 20 Feb 89 17:03:58 EST Date: Mon, 20 Feb 1989 17:02 EST Message-ID: From: Rob Austein To: David Chapman Cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU Subject: Which mailer should digests use? In-reply-to: Msg of 20 Feb 1989 16:36-EST from David Chapman As far as I know, the SIMTEL20 people started doing it unilaterally. There is some justification for it, the idea that the scarcest resource on the entire Internet at one point was (and possibly still is) bandwidth through the "Mailbridge" gateways that connect the ARPANET and the MILNET. So they're sending only one copy of everything through the gateway and letting MC do all the ARPANET side delivery. I suppose if we were organized enough to figure out which addresses in our mailing lists were MILNET side we could route all that traffic through SIMTEL20 for the same reason. (It would also be interesting to see how they reacted with the shoe on the other foot.) JTW and I talked about this a little after it became clear what was going on and we sort of vaugely decided that since SIMLTEL20 and MC are both in the business of providing mail services to people for free we should do what we could to accomodate them as long as we could. I don't think we ever claimed that MC was a network wide list server. I think we did claim that it was a good place to give homes to lists that would otherwise be orphaned by fascist administrative policy on other machines (including our own). I think we've been doing that. I find the fact that the SIMTEL20 people set this up without asking permission somewhat rude, but the number of sites that are providing this kind of free service to people is small enough that I think we should probably all hang together if we don't want to hang separately. The digest stuff, on the other hand, is simply a silly use of the resources that we already have. COMSAT BULK is nearly idle, and the digests are the one thing that could easily be moved there. --Rob  Received: from LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 20 Feb 89 17:01:21 EST Date: Mon, 20 Feb 1989 17:00 EST Message-ID: From: JTW@LCS.MIT.EDU To: David Chapman Cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU, SRA@MC.LCS.MIT.EDU Subject: Which mailer should digests use? In-reply-to: Msg of 20 Feb 1989 16:36-EST from David Chapman From: David Chapman Re: Which mailer should digests use? How did the SIMTEL20 business start? Did they just unilaterally start doing it, or did they ask someone? I seem to recall that when MC as set up there was some talk of it's being a network-wide list server resource, which is absurd in retrospect, but maybe they latched on to that? Actually, I don't think it's all that absurd, and the fact that some of the people in lcs think that's what it is is a big part of why we can occasionally get them to, how you say, pay for it (with space, or repairs, or air conditioning, or not getting all bent out of shape when "people" waste their time frotzing with it, or whatever). The truth of the matter is this - the whole ITS mail situation is on the verge of collapse anyway. To continue offering even minimal service after April 1 we have to get TCP running on the local networks, which means a) ethernet, b) IP hacked at least enough to understand subnets, broadcasts, and minimal routing, and c) TCP hacked enough to at least support reasonable retransmission calculations, and preferably all of the spiffy new algorithms that make it actually work in overloaded and fast-net-gatewayed-to-slow-net situations. To offer reasonable service, there's the whole issue of domain resolution and MX support and better queueing and and and blah blah blah plugh. Amazingly, there's some good news. In most cases the new NSFNET paths to things are proving to be substantially faster and more reliable than the current Arpanet paths. So (given that we (euphemism) actually make that April 1 deadline, and specially if all the TCP changes make it) there's a good possibility that mailer performance is going to get quite a bit better. If not, we're out of business anyway. So lets sit tight until then, and see what happens. -john  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 20 Feb 89 16:36:25 EST Date: Mon, 20 Feb 89 16:36:26 EST From: David Chapman Subject: Which mailer should digests use? To: SRA@MC.LCS.MIT.EDU cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU In-reply-to: Msg of Sat 18 Feb 89 05:16:24 EST from Rob Austein Message-ID: <541394.890220.ZVONA@AI.AI.MIT.EDU> How did the SIMTEL20 business start? Did they just unilaterally start doing it, or did they ask someone? I seem to recall that when MC as set up there was some talk of it's being a network-wide list server resource, which is absurd in retrospect, but maybe they latched on to that? My vote, anyway, is that if something has to give, that's where it should happen. Very few of the SIMTEL list addressees are within MIT (let alone ai/lcs). If, as we just established, MC's mailer is a slow one, there's just no reason at all we should be doing these guys this service. If we agree that's the right thing, I'm willing to play the heavy and take the flack (if that's what is holding us back). Bulking digests seems reasonable to me, so long as digest mail gets delivered reliably. (I still haven't been able to quite understand the screw conditions.) How would the bug lists work, for instance? My guess from watching the queue a lot is that locally-generated digest mail is a lot less than 50%, but it would certainly help. --David  Received: from LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 18 Feb 89 15:52:44 EST Date: Sat, 18 Feb 1989 15:52 EST Message-ID: From: Rob Austein To: JTW@LCS.MIT.EDU Cc: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU Subject: Which mailer should digests use? In-reply-to: Msg of 18 Feb 1989 13:10-EST from JTW@LCS.MIT.EDU Date: Saturday, 18 February 1989 13:10-EST From: JTW@LCS.MIT.EDU I'd like to understand where all the COMSAT time is going before we do this. We keep talking about how overloaded MC is, but sometimes I get the feeling that we just have a very low-capacity mailer. Well, yeah, it is a low capacity mailer, we knew that, it's only got one delivery thread and it's running on top of slow network code. Its major saving grace is that it has some major hair to maximize use of a connection once it's acquired it, but these days as often as not the network or some other randomness nullifies this benefit by breaking the connection before it finishes dequeuing to that host. How many digests are there? Half a dozen at present. See MC:GUMBY;DIGEST >. If we move them -all- to BULK, do you think the load will split closer to 50-50, or will we just kill BULK instead of the normal COMSAT? (Presumably related to how much traffic Wancho is forwarding through MC...) My guess is closer to 50-50. I think the normal traffic is quite sufficient to keep regular COMSAT occupied, even without Wancho. Where does the time go? Munging the queue? Waiting for TCP conns? Waiting for TCP conns, mostly. Look at the STATS files, they're timestamped. Part of what I'm interested in is to see how well COMSAT performs if we remove enough of the load that it might occasionally be able to try dequeuing things because it's time to so rather than because it's been time to do so for 12 hours but it finally got to that point in the queue.  Received: from LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 18 Feb 89 13:10:42 EST Date: Sat, 18 Feb 1989 13:10 EST Message-ID: From: JTW@LCS.MIT.EDU To: Rob Austein CC: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU Subject: Which mailer should digests use? In-reply-to: Msg of 18 Feb 1989 05:16-EST from Rob Austein I'd like to understand where all the COMSAT time is going befor we do this. We keep talking about how overloaded MC is, but sometimes I get the feeling that we just have a very low-capacity mailer. How many digests are there? If we move them -all- to BULK, do you think the load will split closer to 50-50, or will we just kill BULK instead of the normal COMSAT? (Presumably related to how much traffic Wancho is forwarding through MC...) Where does the time go? Munging the queue? Waiting for TCP conns? -john  Date: Sat, 18 Feb 89 05:16:24 EST From: Rob Austein Subject: Which mailer should digests use? To: BUG-DIGEST@MC.LCS.MIT.EDU, POSTMASTER@MC.LCS.MIT.EDU Message-ID: <541474.890218.SRA@MC.LCS.MIT.EDU> I hereby formally (whatever that means) propose that all digests be moved to MC's BULK COMSAT. There are several reasons for this: 1) As things stand, MC's regular COMSAT is guaranteed to be tied up for several hours after the midnight digestifiers run. This is a pain. It may not be common knowledge, but MC is the backup (after XX) for LCS's attempt at a lab-wide mailsystem, so people who pay the hardware repair bills do actually care about timeliness, somewhat. 2) MC's regular COMSAT has been running overloaded for a long time now. STATS files get truncated when Alan or Penny guns COMSAT and restarts it or when the machine reboots, almost never for any other reason. I don't know when the last time was I saw LISTS MSGS under 2000 blocks. By contrast, BULK COMSAT's LISTS MSGS file is 23 blocks long, and all of its OSTATS files are 51 blocks long. It would be nice if we could fix things so that mail would continue to work if both Alan and Penny go on vacation for two weeks. 3) We seem to have tacitly accepted Wancho and company's use of MC as a relay machine, by refusing to do anything sufficiently rude to discourage them. There is no good way presently available to reroute that traffic into the BULK queue. The digests, on the other hand, would be trivial to reroute. 4) Given the relative loading of the two mailers, I think that we would actually be improving the quality of the service to the digest recipients if we moved their traffic to the BULK COMSAT. Right now the only things using BULK COMSAT are AILIST and ARPANET-BBOARDS. The only drawback I can think of is that one has to be a little more careful in setting up the bug addresses for the digests, due to the known locking problems between the two COMSATs. Big deal. Digests are already a special case to set up, one more bit of hair isn't going to change anything. Votes? Dead silence will be taken as meaning that if I go and do this in a fit of annoyance at clogged mail queues, nobody will mind. --Rob  Date: Wed, 16 Nov 88 15:10:23 EST From: Scheme Requestee Subject: Scheme Digest #6 To: mohan%hpljsm@HPLABS.HP.COM cc: BUG-DIGEST@MC.LCS.MIT.EDU In-reply-to: Msg of Fri 11 Nov 88 09:15:36 pst from Joe Mohan Message-ID: <505644.881116.SCHREQ@MC.LCS.MIT.EDU> Date: Fri, 11 Nov 88 09:15:36 pst From: Joe Mohan To: Scheme-request@mc.lcs.mit.edu Re: Scheme Digest #6 Full-Name: Joe Mohan I like the idea of a digest, provided there is some clumping of messages. I would like it even better, if there is some classification, as in AI Digest these days. But individual messages wrapped in a digest header makes no sense. How about at least adjusting the frequency so that we have say 5 to 10 messages in each digest. Seems like a weekly digest will work just fine. This is all reasonable (although I think one week wouldn't be fast enough turnaround), but unfortunately the software that does the digesting doesn't have these features, and no human around here is about to spend the time to do manual digest editing. I could talk to the people who wrote the digester about clumping messages together, but I don't think they'll be very sympathetic. Even if there is only one message in a digest, it still makes sense for the digesting software to deal with the list since then messages will be sent in the middle of the night instead of at prime time. (Another benefit is that the message will be seen elsewhere as having been sent by scheme-request@mc, so error reports will go there instead of to the originator of the message.) The digest was instituted because the MC mailer was becoming overloaded, and has already helped quite a bit. To have a digest consisting of one message may seem weird, but it's uniform and simple. - Jonathan Rees  Received: from SRI-NIC.ARPA (TCP 1200000063) by MC.LCS.MIT.EDU 21 Jun 88 19:27:29 EDT Date: Tue, 21 Jun 88 16:26:20 PDT From: Ken Harrenstien Subject: Re: I put OLD-OZ and MX in MC's COMSATs' HFLUSH lists To: Alan@AI.AI.MIT.EDU, Moon@STONY-BROOK.SCRC.SYMBOLICS.COM cc: JTW@XX.LCS.MIT.EDU, SRA@mc.lcs.mit.edu, ZVONA@AI.AI.MIT.EDU, BUG-COMSAT@mc.lcs.mit.edu, bug-mail@mc.lcs.mit.edu, bug-digest@mc.lcs.mit.edu, KLH@SRI-NIC.ARPA In-Reply-To: <19880621200355.0.ALAN@PIGPEN.AI.MIT.EDU> Message-ID: <12408317967.36.KLH@SRI-NIC.ARPA> Hi. I just dipped into my local KLH-ITS mail file which has been accumulating stuff forwarded from AI/MC (I gave up trying to telnet-read mail at MIT long ago), and have a couple comments about the bulk mailing stuff. (1) Good idea. Even better idea if COMSAT could split its load up automatically and farm it out to subsidiary unqueuers. Hmmm, maybe a good thesis project... (2) Local mail delivery can be handled by locking. That is, when COMSAT attempts to write to a mail file, it should seize the lock and if it doesn't get it just waits until it's free -- normally this doesn't take much time, so it's probably fine just to wait rather than trying to do something else (which would be too hairy). I don't think locking is too hard to enforce. Local mail file writing is concentrated in just one or two places. Even if you can't figure out a simple way to do file locking (e.g. opening for append, but without actually writing anything), you can always just use a single global lock, perhaps in the LOCK file (you'll need to make a few changes to the way this file is referenced and used if you want all of the COMSATs to share it). Good luck. -------  Received: from MCC.COM (TCP 1200600076) by MC.LCS.MIT.EDU 21 Jun 88 16:09:53 EDT Received: from BRAHMA.ACA.MCC.COM by MCC.COM with TCP/SMTP; Tue 21 Jun 88 15:08:51-CDT Date: Tue, 21 Jun 88 15:08 CDT From: David Vinayak Wallace Subject: digest size To: Chapman.pa@Xerox.COM cc: BUG-digest@MC.LCS.MIT.EDU, DANA@AI.AI.MIT.EDU In-Reply-To: <19880621191420.3.ZVONA@SPIFF.parc.xerox.com> Message-ID: <880621150846.8.GUMBY@BRAHMA.ACA.MCC.COM> Date: Tue, 21 Jun 88 12:14 PDT From: Chapman.pa@Xerox.COM Uh, the offending digest was nowhere near 100K. So let's not worry about this yet. (The only way this could really be useful would be for the digester to ask COMSAT how big a message COMSAT was currently willing to deal with, and that's clearly too much work.) Any self-respecting mailer should deal with 100K (comsat itself has a self-image problem -- memory image, that is), but maybe 64K is better. DIGEST shouldn't ask COMSAT -- COMSAT should check and gun itself if the freespace gets below a certain level.  Received: from REAGAN.AI.MIT.EDU (CHAOS 13065) by MC.LCS.MIT.EDU 21 Jun 88 16:07:02 EDT Received: from PIGPEN.AI.MIT.EDU by REAGAN.AI.MIT.EDU via CHAOS with CHAOS-MAIL id 120878; Tue 21-Jun-88 16:04:00 EDT Date: Tue, 21 Jun 88 16:03 EDT From: Alan Bawden Subject: I put OLD-OZ and MX in MC's COMSATs' HFLUSH lists To: Moon@STONY-BROOK.SCRC.SYMBOLICS.COM cc: JTW@XX.LCS.MIT.EDU, SRA@MC.LCS.MIT.EDU, ZVONA@AI.AI.MIT.EDU, BUG-COMSAT@MC.LCS.MIT.EDU, bug-mail@MC.LCS.MIT.EDU, bug-digest@MC.LCS.MIT.EDU In-Reply-To: <19880621174539.2.MOON@EUPHRATES.SCRC.Symbolics.COM> Message-ID: <19880621200355.0.ALAN@PIGPEN.AI.MIT.EDU> Date: Tue, 21 Jun 88 13:45 EDT From: David A. Moon I don't see why you don't just make the bulk mailer never, ever write a local file, and instead ... I didn't do it because I would have had to understand a fair amount more about COMSAT first. Everyone should please recall that I once swore that if I was ever forced to start learning about how COMSAT worked, I would simply terminate mail service on ITS instead. I've already bent that rule by doing what I have done, and I did spend several hours trying to produce a COMSAT that would not do local delivery, and I couldn't find a way to do it.  Received: from Xerox.COM (TCP 1500006350) by MC.LCS.MIT.EDU 21 Jun 88 15:16:55 EDT Received: from Semillon.ms by ArpaGateway.ms ; 21 JUN 88 12:15:28 PDT Date: Tue, 21 Jun 88 12:14 PDT From: Chapman.pa@Xerox.COM Subject: Re: digest size To: Gumby@MCC.COM cc: BUG-digest@MC.LCS.MIT.EDU, DANA@AI.AI.MIT.EDU In-Reply-To: <880620210207.8.GUMBY@BRAHMA.ACA.MCC.COM> Message-ID: <19880621191420.3.ZVONA@SPIFF.parc.xerox.com> Line-fold: no Uh, the offending digest was nowhere near 100K. So let's not worry about this yet. (The only way this could really be useful would be for the digester to ask COMSAT how big a message COMSAT was currently willing to deal with, and that's clearly too much work.) -------  Received: from STONY-BROOK.SCRC.Symbolics.COM (TCP 20024224620) by MC.LCS.MIT.EDU 21 Jun 88 13:47:21 EDT Received: from EUPHRATES.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via CHAOS with CHAOS-MAIL id 422745; Tue 21-Jun-88 13:45:51 EDT Date: Tue, 21 Jun 88 13:45 EDT From: David A. Moon Subject: I put OLD-OZ and MX in MC's COMSATs' HFLUSH lists To: Alan Bawden cc: JTW@XX.LCS.MIT.EDU, SRA@MC.LCS.MIT.EDU, ZVONA@AI.AI.MIT.EDU, BUG-COMSAT@MC.LCS.MIT.EDU, bug-mail@MC.LCS.MIT.EDU, bug-digest@MC.LCS.MIT.EDU In-Reply-To: <19880621021223.9.ALAN@PIGPEN.AI.MIT.EDU> Message-ID: <19880621174539.2.MOON@EUPHRATES.SCRC.Symbolics.COM> I don't see why you don't just make the bulk mailer never, ever write a local file, and instead send all such requests over to the other Comsat (through the network) and have it write the file. This would even include the AILIST archive file. Then there would have to be such fear of screwing things up when making a mailing list be distributed by the bulk mailer. I've only thought of two problems with this. One is the purely clerical problem of finding all the places in Comsat that know about delivery to the local host and turning them off for the bulk mailer, so the network will still be used. The other is to make sure that when mail is sent to the other Comsat for delivery to a local user's mailbox, the address transmitted is the user's name rather than the file name of the user's mailbox, to avoid the problem you mentioned that is caused by the two Comsats having different NAMES files. I think a good way to cope with this would be for the bulk mailer never to do any user-name-to-mailbox expansion through INQUIR, the tourist user directory database, etc. -- any address reaching that point would just be transmitted to the other Comsat. The only address expansion done by the bulk mailer would be done via its NAMES file, and the only kind of delivery done would be network delivery.  Received: from REAGAN.AI.MIT.EDU (CHAOS 13065) by MC.LCS.MIT.EDU 20 Jun 88 22:14:11 EDT Received: from PIGPEN.AI.MIT.EDU by REAGAN.AI.MIT.EDU via CHAOS with CHAOS-MAIL id 120714; Mon 20-Jun-88 22:12:28 EDT Date: Mon, 20 Jun 88 22:12 EDT From: Alan Bawden Subject: I put OLD-OZ and MX in MC's COMSATs' HFLUSH lists To: JTW@XX.LCS.MIT.EDU, SRA@MC.LCS.MIT.EDU cc: ZVONA@AI.AI.MIT.EDU, BUG-COMSAT@MC.LCS.MIT.EDU, bug-mail@MC.LCS.MIT.EDU, bug-digest@MC.LCS.MIT.EDU In-Reply-To: , The message of 19 Jun 88 17:33 EDT from JTW@XX.LCS.MIT.EDU, <438861.880619.ZVONA@MC.LCS.MIT.EDU>, The message of 19 Jun 88 22:25 EDT from ZVONA@MC.LCS.MIT.EDU, The message of 19 Jun 88 22:25 EDT from David Chapman, <433170.880608.SRA@MC.LCS.MIT.EDU>, <12404850562.46.SRA@XX.LCS.MIT.EDU> Message-ID: <19880621021223.9.ALAN@PIGPEN.AI.MIT.EDU> Date: Sun, 19 Jun 88 22:25 EDT From: David Chapman I see that COMSAT BULK is actually running. Would it in fact help with the too-big-mail problem? Should I be using it? Is it reliable? Well, just how big was the file that blew out? Clearly it can only help if your digest is still within -some- limit... I think the digestifier really should be keeping a limit on the size of the digest it is willing to create. If the day's inbox doesn't fit in a single digest, it should write the remainder of the inbox into a separate file (with second filename "BACKLOG"?) to appear at the beginning of the next day's digest. Any mailing list that consistently loses ground with this scheme does not deserve our support. Date: Wed, 8 Jun 88 13:45:15 EDT From: Rob Austein ... Alan, IWBNI ... (2) you sent BUG-COMSAT a short note explaining in English just what you ended up doing with all this .BULK. stuff and what the current proceedure is for launching COMSAT, etc. All I did was create a .BULK. directory that contains all the same files as .MAIL.. COMSAT decides a start-up time which directory to use based on its JNAME. If its JNAME is "VDNBRG", then it uses .BULK. (and sets it's JNAME to be "BULK"). If its JNAME is anything else, it uses .MAIL. (and sets its JNAME to be "IV"). There is a CHANNA;RAKASH VDNBRG to be certain that a bulk mailer gets started at ITS boot time. There is a DRAGON;HOURLY VDNBRG to be certain that the bulk mailer doesn't stay dead for more than an hour. Date: Wed 8 Jun 88 13:59:18-EDT From: Rob Austein So, how does anything ever get queued for the BULK mailer? As far as I can tell the only thing currently going there is AILIST. You just have to know to write a request into .BULK.;MAIL >. Currently the only program that knows how to do this is the AIList digestifier, which I believe is a special purpose program that runs on Lisp Machines. I did -not- do anything to prevent the bulk mailer from delivering mail into local mailboxes. If both COMSAT IV and COMSAT BULK try to deliver mail into the same local inbox, mail will be lost. Fortunately, this is not much of an issue, since there are no -users- who keep their inboxes on MC. (AIList makes -one- local delivery, into an archive file that the other COMSAT never touches.) Unfortunately, it is not quite as easy ask you might at first think to determine that mail delivered by the bulk mailer will not go to any local users. For example, suppose user FRED has an INQUIR entry that says to deliver his mail to MC. Then in MC:.MAIL.;NAMES > he says to forward his mail to framistat%octal!dingbat!bang!mit-eddie.arpa@uk.nrl.dialnet.symbolics.com (which is too long to fit in his INQUIR entry). Well, if FRED is on a mailing list expanded by COMSAT BULK, his INQUIR entry will cause the bulk mailer do decide that FRED is a local user, but since COMSAT BULK looks in .BULK.;NAMES > and not .MAIL.;NAMES >, it will not see his forwarding entry, and so will deliver mail to GUEST1;FRED MAIL! Currently, the safest way to move a mailing list to BULK delivery is to start by examining the expansion of the mailing list as performed by the regular COMSAT to be certain that you know -all- of the entries in the NAMES file that effect the mailing list. I'll bet the digest program has ".MAIL." wired into it somewhere, but that should be easy to fix so that mailing lists could choose the mailer that they want to use.  Received: from MCC.COM (TCP 1200600076) by MC.LCS.MIT.EDU 20 Jun 88 22:03:19 EDT Received: from BRAHMA.ACA.MCC.COM by MCC.COM with TCP/SMTP; Mon 20 Jun 88 21:02:14-CDT Date: Mon, 20 Jun 88 21:02 CDT From: David Vinayak Wallace Subject: digest size To: David Chapman cc: BUG-digest@MC.LCS.MIT.EDU, DANA@AI.AI.MIT.EDU In-Reply-To: <400030.880619.ZVONA@AI.AI.MIT.EDU> Message-ID: <880620210207.8.GUMBY@BRAHMA.ACA.MCC.COM> Date: Sun, 19 Jun 88 16:40:44 EDT From: David Chapman Right, OK, so there was indeed enough volume on the (digested) paganism list that it went over COMSAT's threshold. This makes the automatic digestifier pretty useless. (Gumby, Dana, has this never happened for you?) Is there anything that can be done about this? The size message COMSAT can deal with is limited mostly by the size of the names database, right? That means that if this second-COMSAT hack were hacked, it would have a tiny NAMES database and so could send much bigger messages, right? The size of largest message COMSAT can deal with is a function of the free space; since there are some storage leaks this size goes down periodically. If comsat is bouncing rediculously small messages, gun it down at a convenient time and relaunch it. Then rename the badreq file(s) to be MAIL files. You can't make them too large; more than 100K causes various hosts to barph (there are some VMS hosts which react by summarily closing the connection, then mail the offending message fragment (the Message So Far) back to the request list. Since COMSAT wasn't able to deliver the message and since the remote host didn't accept it then send a refusal, COMSAT follows the spec and tries to deliver the message again. This fills the -request box with about 200 messages of about 90K each. It wouldn't be too hard to cause the digester to check after scanning each message, and stop if the input file position is over 100K. That would keep each digest under that. I suppose it could then even restart itself. A mere matter of programming. Any volunteers?  Received: from XX.LCS.MIT.EDU (CHAOS 2420) by MC.LCS.MIT.EDU 19 Jun 88 17:33:57 EDT Date: Sun, 19 Jun 1988 17:33 EDT Message-ID: From: JTW@XX.LCS.MIT.EDU To: David Chapman cc: bug-mail@MC.LCS.MIT.EDU, bug-digest@MC.LCS.MIT.EDU In-reply-to: Msg of 19 Jun 1988 16:40-EDT from David Chapman The size that comsat can handle is limited dynamically by AvailMem - AllTheRestOfTheJunkInCore. There is a bug in the GC such that this number tends to shrink over time. If you find that you're losing and what you're trying to do isn't absurd, rebooting comsat will often help. Unfortunately I don't have a good feel for what absurd is these days, but I would hope that 100K would be possible. If things start getting this big, other people are going to have problems too, and you might consider sending out smaller digests more often.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 19 Jun 88 16:41:19 EDT Date: Sun, 19 Jun 88 16:40:44 EDT From: David Chapman To: BUG-mail@MC.LCS.MIT.EDU, BUG-digest@MC.LCS.MIT.EDU cc: DANA@AI.AI.MIT.EDU Message-ID: <400030.880619.ZVONA@AI.AI.MIT.EDU> Right, OK, so there was indeed enough volume on the (digested) paganism list that it went over COMSAT's threshold. This makes the automatic digestifier pretty useless. (Gumby, Dana, has this never happened for you?) Is there anything that can be done about this? The size message COMSAT can deal with is limited mostly by the size of the names database, right? That means that if this second-COMSAT hack were hacked, it would have a tiny NAMES database and so could send much bigger messages, right?  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 23 May 88 19:26:44 EDT Date: Mon, 23 May 88 19:24:02 EDT From: David Chapman Subject: Messages got lost? To: Gumby@MCC.COM cc: BUG-digest@MC.LCS.MIT.EDU In-reply-to: Msg of Mon 23 May 88 17:59 CDT from David Vinayak Wallace Message-ID: <383911.880523.ZVONA@AI.AI.MIT.EDU> Oh. His message apparently never got into the digest at all. Probably never got to MC, in fact, because there hasn't been any problem since.  Received: from MCC.COM (TCP 1200600076) by MC.LCS.MIT.EDU 23 May 88 20:05:49 EDT Received: from BRAHMA.ACA.MCC.COM by MCC.COM with TCP/SMTP; Mon 23 May 88 17:59:39-CDT Date: Mon, 23 May 88 17:59 CDT From: David Vinayak Wallace Subject: Messages got lost? To: David Chapman cc: BUG-digest@MC.LCS.MIT.EDU In-Reply-To: <383521.880523.ZVONA@AI.AI.MIT.EDU> Message-ID: <880523175935.2.GUMBY@BRAHMA.ACA.MCC.COM> Date: Mon, 23 May 88 13:06:35 EDT From: David Chapman Perhaps this guy lost because he sent it at 23:00 from the west coast? How would he lose then? 23:00 PST is 02:00 EST. Therefore it becomes an early message in the next day's digest. More insidious is the case where it lies on a gateway (like a UUCP machine which communicates at midnight) and arrives just after the digest has begun to be built. I think a perusal of the Received headers of the incoming messages would show you that this is what happened.  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 23 May 88 13:04:02 EDT Date: Mon, 23 May 88 13:06:35 EDT From: David Chapman Subject: whither administrivia? To: Gumby@MCC.COM cc: BUG-digest@MC.LCS.MIT.EDU In-reply-to: Msg of Thu 19 May 88 17:57 CDT from David Vinayak Wallace Message-ID: <383521.880523.ZVONA@AI.AI.MIT.EDU> I think it would be better if the admin came before the toc, because people are likely to skip over the toc and so miss the admin. Perhaps. This is where it appears in most digests. How can they skip the administriva without running across it? Oh. maybe they use undigestify. It just seems visually hidden. If you don't read the whole toc, in particular, you won't see it. hmm. How about adding (administrivia) to the top of the TOC if appropriate? Good idea. I suspect a lot of people just skip the toc, however. (I do.) How about indenting the admin by four spaces? I think that would set it off visually in a way that would make it stand out. (This is useful only if the toc doesn't run over a screenful...) Does bug-digest go to a file? Now it does: gumby;digest bugs. Perhaps this guy lost because he sent it at 23:00 from the west coast? How would he lose then?  Received: from MCC.COM (TCP 1200600076) by MC.LCS.MIT.EDU 19 May 88 19:00:08 EDT Received: from BRAHMA.ACA.MCC.COM by MCC.COM with TCP/SMTP; Thu 19 May 88 17:57:27-CDT Date: Thu, 19 May 88 17:57 CDT From: David Vinayak Wallace Subject: whither administrivia? To: David Chapman cc: BUG-digest@MC.LCS.MIT.EDU In-Reply-To: <381126.880519.ZVONA@AI.AI.MIT.EDU> Message-ID: <880519175717.2.GUMBY@BRAHMA.ACA.MCC.COM> Date: Thu, 19 May 88 12:47:52 EDT From: David Chapman I think it would be better if the admin came before the toc, because people are likely to skip over the toc and so miss the admin. Perhaps. This is where it appears in most digests. How can they skip the administriva without running across it? Oh. maybe they use undigestify. hmm. How about adding (administrivia) to the top of the TOC if appropriate? Does bug-digest go to a file? (here's an example.) Date: 19 MAY 88 00:06:51 EDT From: Automatic Paganism Digestifier Reply-To: Paganism at mc.lcs.mit.edu To: Paganism at mc.lcs.mit.edu Re: Paganism Digest #5 Paganism Digest #5 19 MAY 88 00:06:51 EDT Today's Topics: More info on the "Ring of Fire" PBS series digestify A new member request, not to go into digest maybe you _can_ have your cake and eat it too Power Animals Merrymeet Announcement and Reg. Form! Biography paganism digest #4 Administrivia: Has anyone besides Brett Slocum sent a message to PAGANISM and had it not appear in the next day's digest? Please let me know if so. Perhaps this guy lost because he sent it at 23:00 from the west coast?  Received: from AI.AI.MIT.EDU (CHAOS 3130) by MC.LCS.MIT.EDU 19 May 88 12:45:48 EDT Date: Thu, 19 May 88 12:47:52 EDT From: David Chapman To: BUG-digest@MC.LCS.MIT.EDU Message-ID: <381126.880519.ZVONA@AI.AI.MIT.EDU> I think it would be better if the admin came before the toc, because people are likely to skip over the toc and so miss the admin. (here's an example.) Date: 19 MAY 88 00:06:51 EDT From: Automatic Paganism Digestifier Reply-To: Paganism at mc.lcs.mit.edu To: Paganism at mc.lcs.mit.edu Re: Paganism Digest #5 Paganism Digest #5 19 MAY 88 00:06:51 EDT Today's Topics: More info on the "Ring of Fire" PBS series digestify A new member request, not to go into digest maybe you _can_ have your cake and eat it too Power Animals Merrymeet Announcement and Reg. Form! Biography paganism digest #4 Administrivia: Has anyone besides Brett Slocum sent a message to PAGANISM and had it not appear in the next day's digest? Please let me know if so. David/paganism-request@mc.lcs.mit.edu ---------------------------------------------------------------------- Date: Tue, 17 May 88 17:19 EDT From: David C. Plummer Subject: More info on the "Ring of Fire" PBS series Message-ID: <19880517211911.9.DCP@SWAN.SCRC.Symbolics.COM> This information is from the May 1988 'GBH member's magazine. The _Adventure_ series (of which the _Ring_of_Fire_ is the first four ...etc.  Received: from REAGAN.AI.MIT.EDU (CHAOS 13065) by AI.AI.MIT.EDU 17 May 88 23:42:58 EDT Received: from NULLSTELLENSATZ.AI.MIT.EDU by REAGAN.AI.MIT.EDU via CHAOS with CHAOS-MAIL id 112745; Tue 17-May-88 23:37:51 EDT Date: Tue, 17 May 88 23:37 EDT From: David Chapman To: gumby@AI.AI.MIT.EDU, alan@AI.AI.MIT.EDU, cent@AI.AI.MIT.EDU cc: postmaster@MC.LCS.MIT.EDU, dana@AI.AI.MIT.EDU Message-ID: <19880518033750.1.ZVONA@NULLSTELLENSATZ.AI.MIT.EDU> This is a draft DRAGON;DIGEST ORDER, documenting Gumby's digester. (Is that a rational place to keep the file? Or should it go in .INFO.;?) Comments requested. -*- Text -*- How to digest an ITS mailing list High-volume mailing lists tie up COMSAT, the ITS mailer, in proportion to the number of messages sent, rather than in proportion to the total number of characters sent. For example, delivering a message to AI-List takes two and a half hours! Mail to hosts that are neither local to MIT nor directly connected to the central nodes of the arpanet -- ``weird'' hosts -- is particularly expensive. Any mailing list which gets more than a few messages per day and which goes to more than about ten weird hosts is going to impose a lot of overhead on COMSAT. Also, the overhead of reading mail from a list is a function of the number of messages received, as well as the number of characters. For these two reasons, large mailing lists should be >digested<; that is, all the messages sent to it during each day are bundled up and sent out as a single, long message. This message is sent in a special format which can be >undigested< -- broken out into individual messages -- by many mail-reading programs. This file explains how to have a mailing list be digested automatically. We'll take as a running example a hypothetical mailing list called FOOBLATZ. All digested ITS mailing lists must reside on MC.LCS.MIT.EDU. You can make forwarding pointers from AI.AI.MIT.EDU or other ITS machines by putting a line like (FOOBLATZ (EQV-LIST FOOBLATZ@MC)) in the machine's .MAIL.;NAMES file. You don't need an account on MC in order to digest a list. All the relevant manipulations can be done remotely from AI, because ITS supports a virtual machine-independent file system. To read and write files on MC, simply prepend MC: to the file name. For example, you can modify MC's NAMES database by reading MC:.MAIL.;NAMES into an EMACS, modifying it, and saving it with c-X c-S in the usual way. For reasons that will presently be explained, a digested mailing list must have an associated home directory on MC (which could be used for other things as well) called the >sname<, and an associated one-to-six-character >jname<. The jname is not necessarily the name of the list, if the list has a long name; it should be a version of it that fits in six characters. FOOBLATZ might have as sname COMAIL; (which is a general large-list directory on MC) and as jname FOO. There's two parts to causing a mailing list to be digested. The first part is that the mailing list entry in MC:.MAIL.;NAMES > should be split in two. In our example, mail sent to FOOBLATZ should be sent collected into a file for later digestion, rather than actually sent out to recipients of the mailing list. This file is called MC:sname;jname INBOX; in our example, that's MC:COMAIL;FOO INBOX. Mail sent to this file should have R-OPTION FAST-APPEND, so the NAMES entry on MC will look like this: (FOOBLATZ (EQV-LIST ([COMAIL;FOO INBOX] (R-OPTION FAST-APPEND)))) The digester will snarf mail out of the inbox and digest it and send it to a >different< list, which has the actual list subscribers on it. We could call this list FOOBLATZ-REALLY: (FOOBLATZ-REALLY (EQV-LIST (@FILE [COMAIL;FOO LIST]))) The digester does some magic to make it look as if the digested message is being sent to FOOBLATZ, even though it is actually sent to FOOBLATZ-REALLY. That way your subscribers will never know the name FOOBLATZ-REALLY, and can't screw things up by sending to it directly. Note here that the mailing list is specified indirectly via a file. This is a good idea for large mailing lists that change frequently, since you don't have to mess around with the NAMED ERR file. The second part of setting up a digested list is that you need to tell the system that the list will be digested. The first step of that is to put your mailing list in the table of digested lists. This table lives in the file MC:GUMBY;DIGEST >, which is actually the MIDAS (assembly language) source for the digesting program. The table is more or less the first thing in the file; it looks like this: ;;; JNAME, SNAME, "PNAME", "RCPT", "RPLTo" define Lists List GDFLMS,GDEAD, "Dead-Flames", "dead-flames-humans", "Dead-Flames" List AVIATN,AVIATN,"Aviation", "aviation-digest", "Aviation" ... termin Each line defines one mailing list. The JNAME and SNAME we've already explained. Spaces count in these fields count, so don't put them in. Spaces outside the quotation marks don't count in the other fields, so put them in so the columns line up nicely. The PNAME is the actual name of the list, FOOBLATZ. The RCPT is the list that digested messages get sent to; FOOBLATZ-REALLY in this case. The RPLTo is the address put in the Reply-to: field that gets sent out on the digest. This will normally be the same list as the address for incoming mail, namely FOOBLATZ. So the line we want to add to the table is List FOO,FOO, "Fooblatz", "fooblatz-really", "Fooblatz" Be careful when messing with the DIGEST source file. If you screw up, mail sent to all the digested lists will lose. Once you've added your list to the table, you need to recompile the DIGEST source file. To do that, type :MIDAS MC:GUMBY;DIGEST_ to DDT (the ITS executive). (Note the underbar at the end.) It will print some random lines of information. If any of them looks like an error message, do not procede; send mail to BUG-DIGEST@MC.LCS.MIT.EDU, asking for help. Otherwise, you've successfully created a file GUMBY;DIGEST BIN, which is a new digester that knows about your mailing list. This object code file needs to be run by a system dragon (demon job) every day at midnight. The DRAGON; directory contains magic that spawns dragons. Two steps are required. First, you need to move DIGEST BIN to the DRAGON; directory: :COPY MC:GUMBY;DIGEST BIN,MC:DRAGON; Second, you need to tell the system to run this BIN on a daily basis. The incantation for this is :LINK MC:DRAGON;DAILY jname,MC:DRAGON;DIGEST BIN where jname is the JNAME of your list; in our case :LINK MC:DRAGON;DAILY FOO,DRAGON;MC:DIGEST BIN This completes the process of making a list be digested. If you run into any problems, send mail to BUG-DIGEST@MC.LCS.MIT.EDU. Three other files are magic for the digester. You can put something in the file sname;jname ADMIN (MC:COMAIL;FOO ADMIN, for instance), and the digester will put it right after the table of contents in the next digest. (Then it'll delete the ADMIN file.) This is useful for making administrivial announcements. sname;jname NUMBER (MC:COMAIL;FOO NUMBER) contains the issue number, in in ascii, of the current issue. If this file does not exist, the digester assumes issue number 1 and creates the file. If there is a problem parsing this file, it is renamed to sname;jname BADNUM, and the issue number is whatever was parsed, with "E" appended (e.g. "259E"). The digester renames the INBOX to sname;jname > (MC:COMAIL;FOO >, where > is actually some integer) as it works on it. If it wins, it deletes that file. If it crashes, the file is left around so you can recover the day's mail.