I'm getting complaints on several of our lists that LISTSERV is issuing duplicate copies of list mail (or digests) to the users. What could be causing this?


LISTSERV probably isn't doing this (in fact, it's almost impossible for LISTSERV to be doing this), but to be sure you should check the daily log for any discrepancy. The most likely causes of this problem are: 


•A broken unix gateway somewhere between LISTSERV and the recipient that is sending the duplicate mail (this has to do with how unix Sendmail recovers from crashes). Generally this problem will appear and disappear, although it's entirely possible for it to go on for days or weeks, depending on how broken the gateway is.

•The SMTP protocol makes it possible for messages to be duplicated if the remote MTA abruptly drops the connection before it sends your local MTA an acknowledgement that it successfully received the message for delivery (via a 250 reply message). When this happens the protocol dictates that the local MTA try to deliver the message again. There is no way to stop this on the sending end short of removing the message manually from the MTA's queue (by the time this happens, LISTSERV has surrendered control of the message to sendmail or LSMTP or whatever MTA is handling the outgoing mail).

•SMTP synchronization problems as described in RFC1047. RFC1047 describes nearly the same problem described in the preceding paragraph but is related more to connections timing out after the "." signifying the end of the message body arrives but before the 250 reply is sent, rather than being due to the remote host simply cutting the connection before sending it. The former may be indicative of a software or machine resource problem, whereas the latter may be due to extensive "sophisticated processing of the message, in an attempt to confirm that they can deliver the message" (RFC1047, page 2). A solution to the problem as described by RFC1047 may be to raise your MTA's timeout for the site in question, perhaps from 5 to 10 minutes.


In the first three instances, the only way to find the point of duplication is to compare the date stamps on the various "Received:" header lines in the duplicated mail. The point at which they no longer match is where the duplication is taking place.


•Look for users who may have an account on one host .forwarded to a second host but still have subscriptions for both accounts. (This last is extremely common in these days when people jump from ISP to ISP almost on a whim.) 

•Finally, duplicates can be generated when the SMTP server on the receiving end does not respond to the end-of-data signal from the sending server and times out the connection. Per standard, when a connection times out, the sending server must assume that the message was not received properly on the remote side. Since the sending server cannot know if the message was actually received in its entirety, the message must be requeued for delivery. If the remote side server continues to exhibit this behavior, the sending server will continue to redeliver the message until it reaches its retry limit.

•[Very old FAQ regarding the Cisco PIX firewall removed as being obsolete.]