Release Notes for LISTSERV® Version 15.0

 

Copyright © 2006 L-Soft international, Inc.

3/5/2010 10:05 AM

 

 

IMPORTANT: LISTSERV 15.0 requires a version 15.0 LAK!

 

You must obtain and install a LISTSERV version 15.0 product LAK prior to upgrading your server, or LISTSERV will not start after the upgrade.

 

More information

 

 

Go directly to the table of contents

 

 

CONTENTS

 

USABILITY: New and enhanced web interface design

 

USABILITY: Tight integration with LISTSERV Maestro 3.0

 

USABILITY: Background DISTRIBUTE delivery

 

USABILITY: [HPO] DISTRIBUTE worker tuning

 

USABILITY: [HPO, Non-VM] Rate-limiting delivery pools

 

USABILITY: [Non-VM] QUERY FILE command available

 

WA: Current Version

 

OTHER: Miscellaneous Changes and Fixes

 

Important OS Support Information

 

Applying LISTSERV 15.0

 

 

USABILITY

 

LISTSERV is a well established email list management product and the de facto industry standard. The

software has always been renowned for its reliability, scalability and industry-leading feature set. As the

importance of usability has grown, it has become an integral part of any development, especially when

end users range from beginners to experts. The new LISTSERV 15.0 interface was created with a

primary focus on usability.

 

The development team started out by analyzing the strengths and weaknesses of the existing Web

interface and deciding on an improved navigational system. Technical experts and usability designers

worked together on creating the best navigational structure that would provide quick and easy access to

the different sections throughout the Web interface. Many existing sections were overhauled to make

them easier to digest. The challenge was to provide users with as much power and functionality as

possible without overwhelming.

 

Several sections were divided into tabs, which provide quick access and present information in a more

organized manner. Three modes -- Basic, Expert and Tutorial -- were created to allow users to tailor the

Web interface according to their expertise level. More advanced features are only shown in Expert Mode,

while instructions and tips only appear in Tutorial Mode.

 

A test group, ranging from LISTSERV experts to total beginners has tested the new user interface using a

test method based on heuristic evaluation and usability checklists. Heuristic evaluation is a usability

engineering method for finding the usability problems in a user interface design. Heuristic evaluation

involves having testers examine the interface and judge its compliance in regard to recognized usability

principles. After careful consideration and adjustments based on tester feedback, we are hopeful to have

crafted a user interface that provides the best of both worlds, usability and functionality.

 

Project Coordinator

Sandra Rudenmo

 

New and enhanced web interface design

 

LISTSERV 14.5 administrators got a taste of what the upcoming 15.0 web interface would look like in the Deliverability Assessment tool introduced in that version.

 

Now LISTSERV 15.0 integrates that same look and feel into an entirely new, modern, and easily navigable web interface.  The enhancements require the latest version of the WA CGI stage and new default templates, and include

 

  • Server Administration and List Owner dashboards provide at-a-glance summaries
  • LISTSERV Maestro sign-in integration
  • Users can set persistent personal preferences -- preferred start page, experience mode (expert, basic, and tutorial), text size, navigation style, and archive preferences.
  • Site configuration changes can now be made "on the fly" from a new site configuration wizard.  Certain changes may require a LISTSERV restart; others take effect immediately.
  • The List Creation wizard has been substatially owverhauled and improved.
  • The List Moderation tool has been improved.
  • The list archive browsing interface now shows HTML messages in their "full glory".
  • List archive management is now possible through the web interface.  List owners may now delete individual messages from the archives.
  • A new "Template Edit" mode is available to administrators and list owners.  This mode is reached by clicking the "Edit Page" link found at the top right of the page when the page is viewed normally. ("Edit Page" links appear on a given page only for those users who are authorized to edit that page.) Template Edit mode greatly simplifies the process of site-level or list-level customization by displaying each element of the page along with individual links to edit those elements,

 

Sites with heavy customization that are not ready to migrate to the new templates will be pleased to know that their existing templates will work with LISTSERV 15.0 and the latest WA.

Tight integration with LISTSERV Maestro 3.0

 

LISTSERV Maestro is a highly-intuitive, advanced e-mail communication solution.  It allows you to manage and track sophisticated, database-driven opt-in email campaigns with a powerful, yet easy-to use Web interface.

 

LISTSERV Maestro has evolved significantly from Version 1, a user-friendly interface to LISTSERV's DISTRIBUTE command, to Version 2, to which was added a data warehouse to collect and store information for both itself and for LISTSERV mailing lists, and an easy-to-use wizard for creating simple database-backed LISTSERV lists within Maestro itself.

 

Now comes LISTSERV 15.0 and LISTSERV Maestro 3.0.  The two interfaces have been completely redesigned, merging the best features of each into the other, and allowing free flow from one to the other.

 

While of course LISTSERV 15.0 can still function independently of Maestro, the two products together offer the most comprehensive email communications package on the market.  With the recent addition of HDMail, L-Soft's new High Deliverability mail engine, the L-Soft software suite has become an email communications solution unmatched in deliverability, speed, and ease of use.

[HPO] Background DISTRIBUTE delivery

 

A new background DISTRIBUTE delivery feature has been implemented for high-volume LISTSERV HPO sites.  This feature is meant especially for sites signing messages with DKIM, but also for sites using Embedded Mail Merge (EMM) that regularly send jobs larger than 10K recipients.  Such sites may require faster processing in order to accommodate other jobs and interactive requests in a timely manner, but by the same token may not need the power of the dedicated DISTRIBUTE workers introduced in LISTSERV 14.4.

 

Background DISTRIBUTE is enabled by default for all LISTSERV Classic HPO 15.0 sites.  It can be disabled if necessary by setting DEFAULT_DIST_BACKGROUND to 0 in the site configuration file and restarting LISTSERV:

 

VM:

DEFAULT_DIST_BACKGROUND = 0

VMS:

DEFAULT_DIST_BACKGROUND "0"

unix:

DEFAULT_DIST_BACKGROUND=0

export DEFAULT_DIST_BACKGROUND

Win:

DEFAULT_DIST_BACKGROUND=0

 

 

If the default is changed, background DISTRIBUTE can be activated per DISTRIBUTE job with the new DISTRIBUTE keyword BACKGROUND=YES.

 

Conversely, if the default is retained, you can specify BACKGROUND=NO explicitly in a DISTRIBUTE job to override the default. This is particularly recommended for test jobs.

 

Instead of creating the outbound messages immediately, LISTSERV saves the relevant data in a background queue. Each data file in the queue contains information for a certain number of recipients.  The default is 10,000 and can be tuned with the new DIST_BACKGROUND_JOBSIZE site configuration variable if required.  However, the default value should be a good balance between not creating too many files, and not creating any huge files that take a long time to open. With this setting, a job with 1M recipients will create only 100 files. Each file is somewhere between 1M and 5M depending on the job.

 

The DISTRIBUTE command completes as soon as the last data file is written (in much the same way as when using DISTRIBUTE workers). In one test on a desktop PC, 10k recipients took 0.3 sec elapsed time.

 

Background jobs are processed in sequence, in the order in which they were created, with only one job active at a time. Background delivery pauses immediately if LISTSERV has any other work to do or any other mail to send, although it should be noted that any mail already spooled will continue to be sent. For efficiency, background delivery will not pause until at least x number of messages have been sent. (This number is set by the new site configuration variable DIST_BACKGROUND_CHUNKSIZE, which defaults to 50.)  Progress messages are printed in the LISTSERV log whenever a data file is finished and LISTSERV switches to the next one.

 

Background distribution is compatible with delivery pools, and with the new SMTP rate limiting feature discussed below.

DISTRIBUTE worker tuning

 

This feature adds a configuration variable that lets you limit the use of DISTRIBUTE workers to oversize messages. If specified, DIST_FORWARD_MINSIZE is the minimum message size, in kilobytes, to make use of the DISTRIBUTE workers. Jobs below that size are processed locally. The message size is defined as the size, counting CRLF terminators, of the contents of the DATA portion of the job.

To forward all messages over 250k to a DISTRIBUTE worker, you would thus use the following:

DIST_FORWARD=[suitable worker pool definition]
DIST_FORWARD_MINSIZE=250
DIST_FORWARD_THRESHOLD=2

The purpose of this feature is to better control bandwidth. All jobs above a certain size can be routed to a separate DISTRIBUTE worker, which in turn might be configured to use a separate mail server with a bandwidth cap.

The DISTRIBUTE worker in question can run on older hardware as long as it has plenty of disk space. It is expected to build a large queue when oversize jobs are processed.

The threshold of 2 allows single-recipient test messages to be processed locally (faster).

[HPO, Non-VM] Rate-limiting delivery pools

 

With non-VM LISTSERV HPO, it is now possible to "rate-limit" LISTSERV's output to its SMTP_FORWARD host(s) by defining limits for individual delivery pools (or server-wide if you do not choose to implement separate delivery pools).  This is done by setting the site configuration variable SMTP_RATE_LIMIT appropriately.  For example, the following setting rate-limits the entire server to 12Mbps of throughput to the outbound forwarding host.

 

VMS:

SMTP_RATE_LIMIT "12Mbps"

unix:

SMTP_RATE_LIMIT="12Mbps"

export SMTP_RATE_LIMIT

Win:

SMTP_RATE_LIMIT=12Mbps

 

Using the feature with delivery pools allows the server administrator to limit the impact that large lists or large DISTRIBUTE jobs have on total throughput from LISTSERV to its outbound MTA(s), reserving sufficient bandwidth so that administrative mail and smaller lists can get through at the same time. Thus a more complicated rate-limiting scheme might be as follows:

 

VMS:

SMTP_RATE_LIMIT "12Mbps AB:1.5Mbps C:150kbps"

unix:

SMTP_RATE_LIMIT="12Mbps AB:1.5Mbps C:150kbps"

export SMTP_RATE_LIMIT

Win:

SMTP_RATE_LIMIT=12Mbps AB:1.5Mbps C:150kbps

 

In this case, the entire server (including existing delivery pools that have not been otherwise specified) is limited to 12Mbps outbound, while pools A and B are limited further to 1.5Mbps and pool C is limited to 150kbps.  Anything not mentioned is unlimited. In the above example, D is technically unlimited, other than for the overall 12Mbps limit for all traffic.

 

Each outbound message belongs to exactly one pool and is subjected to exactly two rate limits (either of which can be unlimited): the pool rate limit and the overall rate limit. Whichever is strictest at this particular moment sets the bar.

The formal syntax of SMTP_RATE_LIMIT is

 

SMTP_RATE_LIMIT=limit1 [limit2 [...]]

limit: [pools:]maxbps

pools: any  valid pool names  (as usual, the  default pool is  '-'). When
       specified,  each listed  pool receives  the specified  limit. When
       omitted, the overall rate limit is defined.

maxbps: a positive number followed by  'kbps', 'Mbps' or 'Gbps' (case not
        material). No space between the number and 'kbps'.

Example:

SMTP_RATE_LIMIT=12Mbps AB:1.5Mbps C:150kbps

Please note carefully the following:

 

  • A LISTSERV HPO LAK is required

 

  • LISTSERV's rate-limiting imperatively requires that asynchronous SMTP "workers" (SMTP_FORWARD_n) be used. Results will be unpredictable if used with the deprecated ASYNCH_SMTP option. The fallback SMTP_FORWARD (synchronous) delivery mechanism ignores rate limits.

 

  • The rate limit is specific to LISTSERV. It can be used to limit the rate at which LISTSERV feeds your outbound SMTP MTA, but it does NOT limit the rate at which your MTA feeds messages to the outside world. Rate-limiting your MTA on the outbound side is beyond the ability of LISTSERV to control.

 

  • Rate limiting does not split messages. Any given message is either sent immediately as it is, or delayed. If you have large attachments with MAXBSMTP=10000, you will not get a smooth ride. If you want a smooth traffic graph on your network monitor, make sure that MAXBSMTP is low enough. This being said, it will work either way, correctly and accurately.

 

  • The limit is applied to the size of the message body (including RFC822 headers and CRLF, but not including the RFC821 envelope). This may require that the rates be tweaked somewhat to get the actual desired performance.

 

  • To keep synchronization overhead to a reasonable level, the rate limits are split among participating workers, which work independently as they always have. A little report is written at the beginning of the SMTP worker log. This means you will probably see somewhat less usage than you have requested, especially if you have workers servicing multiple pools. When worker #1 is servicing a large distribution for pool B, its rate quota for pool A is not being used. Worker #2 has no idea what worker #1 is doing and cannot "borrow" its pool A quota during this time. There is a simple solution to that problem: one worker, one rate-limited pool.

 

  • For reference, kilo, mega and giga as used by SMTP_RATE_LIMIT are powers of 2, not 10.

 [Non-VM] QUERY FILE command available

 

A basic version of QUERY FILE has been implemented for non-VM systems. While this has been done primarily to allow WA to query the presence of DEFAULT.DKIM, it can be used to query for any file known to LISTSERV. The syntax is the same as on VM, and the FLags options is recognized for compatibility, but does nothing.


QUERY FILE DEFAULT.DKIM

 

(abbreviation: Q F)

 

Sample output would be something like


DEFAULT       DKIM            N/A CTL          727 2005-11-25 16:56:25

or:

DEFAULT       DKIM            N/A CTL            . .......... ........

if the file is defined but does not exist.  Note that certain files (including DKIM files) are defined implicitly by LISTSERV and do not necessarily need an explicit entry in a CATALOG or FILELIST file in order for LISTSERV to know about them.

That is the default non-VM query format, but customers with old VM scripts that utilized LISTSERV's file server functions can also choose the old VM query format, which results in the following sample output:

SERVICE  NAMES      ALL LMC V      65   248 05/01/13 15:17:55
DOMAIN   NAMES      ALL LMC .       0     0 ........ ........

Choosing the query format is done by setting the Boolean value VM_STYLE_INDEX in the site configuration (note that this setting centrally affects all file server commands, not just QUERY FILE).  To use the VM query format, set VM_STYLE_INDEX to 1.  To use the default non-VM style format, set VM_STYLE_INDEX to 0 (or remove from, or comment out the variable in, the site configuration file).

 

Examples (setting VM_STYLE_INDEX to 1 for VM-style format):

 

VMS:

VM_STYLE_INDEX "1"

unix:

VM_STYLE_INDEX=1

export VM_STYLE_INDEX

Win:

VM_STYLE_INDEX=1

 

When using the VM-style format, your script should look for eight dots to identify a missing file. 

 

When scripting using either format, note that if your script does not find the line starting with eight dashes, the command failed.

WEB INTERFACE - WA

Current Version

 

LISTSERV 15.0 requires that the Web interface (commonly known as "wa") is version 2.4.0b1 R2338 or later. To determine what version of "wa" you currently have running, browse to http://yourserver/path-to-wa/wa?DEBUG-SHOW-VERSION for unix, or http://yourserver/path-to-wa/wa.exe?DEBUG-SHOW-VERSION for Windows and OpenVMS).

 

In LISTSERV 15.0, the WA executable version is also displayed on the Server Administration Dashboard.

 

OTHER

Miscellaneous Changes and Fixes

 

The following miscellaneous fixes and changes are included in LISTSERV 15.0.

 

  • Previously, the optional SMTP_LISTENER_IP variable could be used to tell SMTPL to listen only on a specific IP address, rather than listening on all available addresses. The syntax has been extended to allow you to specify more than one address to listen on:

    SMTP_LISTENER_IP=a.b.c.d[:port] [a.b.c.d[:port] [...]]

    When no port is specified, the value of SMTP_LISTENER_PORT (default: 25) is used.

Important OS support information

 

A list of operating systems (and versions) under which LISTSERV is supported can be found at:

 

http://www.lsoft.com/products/listserv_os.asp

 

LISTSERV 15.0 is supported only under the operating systems found on that page.

 

Sites running older versions of LISTSERV under operating systems which are no longer supported by L-Soft should migrate to a different operating system. Please contact your sales representative for further information.

 

Sites running the Windows 95 shareware should note that their licenses will not activate the product under Windows XP. Please contact your sales representative for alternatives if you are planning to upgrade to Windows XP (optionally you may migrate to the LISTSERV Lite Free Edition). Sites running the Windows 95 Lite Free Edition can simply upgrade to the Windows NT/2000/XP LISTSERV Lite Free Edition. You may also elect to continue running LISTSERV under Windows 95/98/Me, but there will be no further new versions or fixes for that platform.

 

Applying LISTSERV 15.0

 

IMPORTANT: Install your LISTSERV 15.0 product LAK before upgrading! A valid product LAK (License Activation Key) with "REL=15.0" must be installed before upgrading or LISTSERV will not start after the upgrade.

 

If you have not received a LISTSERV 15.0 product LAK, please contact your sales representative or SALES@LSOFT.COM before upgrading!

 

To find out if you can upgrade to LISTSERV 15.0 with your current license key, please issue a SHOW LICENSE command to LISTSERV and examine the response. It will be similar to this:

 

License type:       Permanent

Expiration date:    None - perpetual license

Maintenance until:  12 October 2007, serial number MNT-XYZ-1

Capacity:           Unlimited

Version:            15.0

Serial number:      XYZ-1

Build date:         1 Nov 2006

 

Your license key will be valid for the 15.0 upgrade if your current LAK is for version 15.0 or higher.

 

Sites running LISTSERV 14.4 or later may use the LAK input tool in the Web Administration interface to apply and check their new LAK before upgrading.

 

The installation kits found on L-Soft's web site can be used either to install a new copy of LISTSERV or to upgrade an existing installation. To download LISTSERV 15.0, simply go to L-Soft's Web site and download an evaluation copy of LISTSERV or LISTSERV Lite, then follow the installation instructions for your operating system. The kits can be found at:

 

http://www.lsoft.com/download/listserv.asp

 

http://www.lsoft.com/download/listservlite.asp

 

SPECIAL NOTES

 

1. This document does not include upgrade instructions. Please see the installation guide specific to your OS platform for upgrade instructions.

 

VMS:                http://www.lsoft.com/manuals/15.0/vmsinst.html

 

unix:                 http://www.lsoft.com/manuals/15.0/unixinst.html

 

Windows:          http://www.lsoft.com/manuals/15.0/ntinst.html

 

VM sites currently at the 1.8e or 14.x level should download ftp://ftp.lsoft.com/listserv/vm/upd150.hex and install it per the "Fixes and Upgrades" section of ftp://ftp.lsoft.com/listserv/vm/00-read.me . VM sites currently at the 1.8d or earlier level must first upgrade to 1.8e (LISTSERV 14) before applying this level set fix.

 

2. The unix kit now only requires a single download. You do not need to download a common.tar.Z file as in previous versions, as those files are now included in the single OS-specific installation file.

 

3. LISTSERV version 15.0 is available only for operating systems currently supported by L-Soft. When browsing FTP.LSOFT.COM, you may find installation kits for other operating systems, such as Ultrix or SunOS 4.x, but these kits will be based on older versions and/or code bases. L-Soft no longer has development systems for unsupported operating systems and is not in a position to compile LISTSERV 15.0 for these systems.