Listproc 8.1 Release Notes
Contents
Smart Loading of List Configurations- In past versions, the Listproc command would load configuration and owner information for all lists at startup. Hence, processing any request - list related or otherwise - would require parsing all of the configuration information, and allocating memory for all of the lists. Listproc 8.1 reduces the extra processing by only loading a list's configuration information if and when it receives a request for that list.
- Owner information is not loaded when commands are sent by the manager, as no checks are required.
- Requests that require that all lists be loaded (like LISTS) do so after all syntax and semantics checks have been made.
- Remote lists loading is done on demand as well. Previously, all remote lists were loaded when a list was not local.
- In previous versions, an INIT LIST-NAME command would cause serverd to throw out all previously loaded list information, and reload everything. Listproc 8.1 only loads the new list information, but does not dump previously configured lists.
- When processing multiple INIT commands submitted in a single email, Listproc only initializes once - AFTER all commands have been processed.
- Listproc 8.0 would not spawn any new list processes while it processed incoming requests. Version 8.1 lifts this restriction.
- The memory footprint for served has been reduced by around 72% - for 1000 lists it used to consume 17MB; it now consumes 2.3MB.
- Special processing has been added to speed up owner logins.
- Subscription managers, errors-to recipients, moderators and subscribers can now specify a list name when logging in to ILP. This reduces the number of checks that must be made, and greatly speeds up the authentication process.
- Improvements have been made to some of the internal string handling routines, cutting their running time in half.
- The Listproc error processing thread now checks periodically to make sure it is not blocking any of the list threads. If it is, it allows the list threads to take precedence.
- A WHICH-OWNED command has been added to allow owners to find out which lists they own.
- The SUBSCRIBE command now behaves differently when the user is already subscribed, but has her/his mail method set to POSTPONE. In the past, the user would simply be informed that they were already subscribed. In addition, Listproc 8.1 resets their mail method to the default for that list, and notifies the user that their mail method has been reset.
- List owners now have the ability to force subscribers to confirm
their intent to either subscribe OR unsubscribe from the list. If
these options are turned on, subscribers who request to be
(un)subscribed will be sent an additional message, that they must
forward back to the server before their request will take effect.
This can help prevent spoofed messages from adding bogus users, or
removing legitimate ones.
The following new options are defined for use with the CONFIG command:
CONFIRM-ALL-SUBSCRIPTIONS DONT-CONFIRM-ALL-SUBSCRIPTIONS CONFIRM-ALL-UNSUBSCRIPTIONS DONT-CONFIRM-ALL-UNSUBSCRIPTIONSBoth of these default to the 8.0 behavior of NOT requiring confirmation.
-
Listproc 8.1 also introduces the ability for subscribers to issue
commands on behalf of a different address. Since subscribers don't
have permission to force actions for other users, these requests
ALWAYS generate a confirmation message to the affected user.
Among other things, this will allow users whose sending address has changed, but who can still receive at their old address, to unsubscribe themselves. For example, user "jrvb@info.cren.net" could send the following message to remove themselves from the FOO list:
unsubscribe foo for jrvb@cren.netA confirmation message would be sent to jrvb@cren.net, asking if he would really like to be removed from the list. If he forwarded this message back to the server, the request would go through. Otherwise, no action would be taken.The decision of whether or not to allow these ".... for user" commands is up to the list owners, and controlled by the following list config options:
ALTERNATE-ADDRESS-COMMANDS NO-ALTERNATE-ADDRESS-COMMANDSThe default is to follow the previous behavior, and NOT allow subscribers to specify an alternate address.
-
List owners now have the option of allowing users to subscribe
without giving a name in the subscribe command. This is controlled
by the following new list config options:
ALLOW-EMPTY-SUBSCRIBER-NAMES DONT-ALLOW-EMPTY-SUBSCRIBER-NAMESIf DONT-ALLOW-EMPTY-SUBSCRIBER-NAMES is set, Listproc will accept commands such as this for the list:SUBSCRIBE listnameIn this case, the name field in the subscribers file will be filled with "No Name Given".The default is to use the 8.0 behavior, and require names for subscription.
-
The new configuration option, REVIEW-BY-SUBSCRIBERS-SHORT, grants
subscribers the ability to retrieve the list's current settings but
NOT to retrieve the list of current subscribers.
-
The MTA-HOST list config option allows the site manager to fine tune
Listproc's performance by splitting the mail delivery load among
multiple hosts. This way, lists that generate a particularly large
amount of traffic won't slow down the delivery of other lists.
The syntax is as follows:
CONFIG <list> <password> MTA-HOST <host> [port]
<host> can be "" or '' to revert to the system default. This option works with the system mailmethod only. The default port is 25. A dash can be used as place holder for the hostname if only the port needs to be changed.
-
The maximum length (in numbers of lines) for message ID and checksum
files is now configurable with the max_file_length directive:
max_file_length nlines
GENERAL CHANGES
- Made sysexec() fault-tolerant with fork() -- retry up to 1 minute
when fork() fails. Usually fork failures are temporary.
- Modified the way Listproc handles addresses to be ignored; before it
would ignore them, but it would report so for every request sent in
thus wasting cycles; it now ignores them after the first line of
input and tosses the rest of the message.
- The REVIEW and STATS defaults are now -TO-OWNERS.
- Added the following regex to SUSP_SUBJECT: MAIL.*[ \t]FAIL Some
weird address, "SJPL, SJPLPO/POSTMASTER" <SJPLS@ci.sj.ca.us>
generated a loop with Subject: Mail Failure that was not caught.
- As a result of the above we have reinstated mailer-daemon checks not
only on the address, but the user name too, and we have expanded the
check to the Resent-From address and user name.
- modified server_response() to read the MTA response all at once, rather than one character at a time. This requires that all systems support non-blocking I/O.
BUG FIXES
- Fixed another core with handling of mmap(); it occurred when the file
size was an exact multiple of the page size.
- Restricted matching of alternate addresses to 9; it is possible to
define an alias like .+ which will match everything and Listproc
will match all addresses thus overflowing internal buffers.
- Fixed a logic bug with sensing requests in list mail (broken if no
ignored_requests were defined in the config file).
- Fixed a long outstanding bug with zero-length temp files left around
in tmp/. Still testing the fix on list.cren.net though (it's a
timing issue)...
- Updated queued to require a semaphore id when using multiple
threads; the semaphore id is passed onto pqueue.
- Fixed bug introduced in 8.0 beta where mail to multiple owners with
any mail method except system was not working correctly (addresses
were mangled). Turned out to be a wider problem with sending to more
than one addresses with these mail methods.
- Fixed bug with 7.2 and prior with mail sent to listname-request
(meant for the owners); Listproc failed to preserve MIME headers of
incoming MIME messages thus sometimes rendering impossible the
decoding of these message by the owners.
- Fixed core with "SET list FOR address"; this was broken in 7.2 too.
- Fixed core dump with fill_text() in list.c that Warren had written;
apparently it did not work with words longer than 79 chars; an
address (considered a word) longer than that would cause a seg
fault.
- Fixed a serious bug with gexit() on most executables: it was calling
internal routines, like tsprintf() and setup_gstring(); but when a
error occurred in one of these gexit() would be called, which would
in turn call these routines again; this internal loop would also
fill up the .warning file and eat up all disk space
eventually. gexit() is now a true exit routine.
- Fixed core dump with multi-line header lines; it was possible for
header lines to get mixed up, or a seg fault to occur.
- Converted variable 'symptoms' to a dynamically allocated string in
list.c is_mail_loop() and in similar places in listproc.c; we found
cases of too many error reports that caused memory overflow and
crash. We now report unique error findings only.
- Fixed a bug where requests spanning multiple lines separated by the
& continuation character were not being properly saved in the batch
queue.
- Fixed bug with sending a partial digest when a user changed mail
mode from DIGEST to ACK or NOACK; when the same list was
distributing its regular digest at the same time havoc could ensue.
- queued now requires both semaphore ids as arguments; it was a bug in
8.0 to require only one.
- Under 8.0 some pqueue processes would not place the outgoing mail in
mqueue/ if delivery failed and would toss the file entirely; this
was due to the missing gatesid that was not passed to them. This is
fixed in 8.1 by fixing list.c and queued (see above).
- Fixed a long lasting bug with sorting of subscribers files; we were
not handling subscriber names separated with tabs correctly; this
resulted in internal sorting errors.
- Fixed bug with SET request where if more params were specified and
'option relaxed_syntax' was set in LPDIR/config, we neglected to
extract only the information we were after and toss the remainder;
we used the whole string with the bogus extra parameters, thus
corrupting the subscribers file.
- Found and fixed file descriptor leak in mmap_owner_listed(). This
leak could potentially bring a system down when the system limit of
open files was reached.
- There are some cases in the code when a string initialized with
strncpy() was not terminated with \0. That could cause core dumps.
- Fixed two more core-dumping problems with the SYSTEM request (new
with 8.1) and a special error condition with list: while
distributing the digest if an invalid address were encountered core
was dumped.
- Fixed core dump with ARCHIVE ... REVIEW of non-list related
archives, issued by archive owners.
- Fixed bug with zero-byte allocation in farch.c.
- Fixed core dump with list and Listproc due to bad treatment of
pointers after realloc() -- realloc() may move the block and we were
not accounting for it in some cases.
- Fixed bug with the load-on-demand stuff -- unix_cmd directives were
not properly loaded.
- Fixed bug in config_owner_prefs (was modified in 8.1) -- it caused a
memory leak and a segmentation violation after repeated INIT
requests.
- Revised signal handling for SIGCHLD in serverd.c and list.c after
proving that signals can get lost; the result was zombie processes,
and infinite wait by parents for processes to exit. This turned up
as more threads were being defined by various sites.
- Fixed bug in ILP parser, to correctly handle extra spaces in archive
requests.
- Now save unprocessed server messages when shutting down.
- Fixed core-dumping bug with mail delivery in 8.0 with all mail
methods aside from system.
Last updated:
June 01, 2007