This is a portability release that adds support for MacOS 10.
My friend Andras Salamon abandoned Linux for the seductive
embrace of a Unix with a working GUI, but discovered that
Postoffice wouldn’t run on it due to the usual
collection of header-file oddities (Darwin is a FreeBSD
derivative, so the published interface for malloc() and
friends lives in stdlib.h instead of malloc.h. But, unlike
FreeBSD, Darwin doesn’t even have a malloc.h, so the build
fell over dead in a fairly predictable fashion. There were
also some resolver portability issues with mx.c that needed
to be #defined into functioning properly.
Aside from this (and the teeny detail that I've not verified it. I've got a MacOS 10 laptop sitting at home, but I don’t have a copy of Xcode to test it out) there are no improvements or bugfixes here.
Valkai Elod sent me email today pointing out that
when I put in immediate delivery, I screwed up the
code that does mail delivery on systems that have a working
flock() system call. I left out two characters in a string,
and that meant that postoffice couldn’t deliver mail anymore.
File this defect under “I am an idiot” and release a new patch!
1.3.8a has one major feature, one minor feature, and one major bugfix. The features have been percolating in my head for a while, and I finally stuffed them in when I discovered a new bug.
The bug is a resolver bug; if a domain has broken reverse dns and has cnamed the ptr record back to itself, postoffice would drop into an infinite loop, hammering the name server for the same cname over and over and over and over. I have “fixed” this by putting in a hard 10-indirections limit for cnames; if the resolver has to go any further in that that it will just not resolve the name.
The minor feature is that I now support a very limited
subset of sendmail’s -d debugging options. It turns
out that some modern versions of the worlds least portable
autoconfig system (and that would be GNU configure) want
to know if there’s a sendmail on the box, and if they find
it they do “sendmail -d0” to find out interesting
information about that version. Well, postoffice
isn’t sendmail, so most of the debugging output is
meaningless, but it should at least not whine about the
-d flag. So -d0 simply spits out the version number
in a more-sendmail format.
The major feature (which is almost enough to roll from 1.3
to 1.4, but I'm saving that for the long-promised cleanup
of the bounce message code) is that I've implemented the
new option “immediate”, which tells postoffice
to attempt to immediately deliver mail to remote recipients.
I don’t have much of a userbase, but most of that userbase
has been fairly persistant in asking for this feature. Like
sendmail, postoffice does remote delivery
by starting a new delivery process, so if you have a lot
of mail going through the system you might not want to do
it, but it appears to be polite on machines with only a
small amount of outgoing traffic.
Shock! Horror! It’s another release that doesn’t include any bugfixes. 1.3.7 has one new feature in it, plus more tweaking to the escape-from code that I started writing in 1.3.6.
The new feature is that authdb blacklisting has been (properly) implemented. The previous implementation blacklisted entries by setting the delay time to MAXINT(timet), which becomes less and less useful as it gets closer to 2038. For 1.3.7, I redid it to use a special blacklist token, so the delay time will always remain insultingly large.
The tweaks to escape-from are to check for the existance of
memstr() before enabling the built-in one, and now all local
deliveries (in 1.3.6, the only place I did from escaping in
was delivery to local mailboxes; 1.3.7 fixes that and also
does from escaping when piping to executables) have embedded
Froms escaped.
escape-from, which tells the local mail delivery
code to scan the incoming message for lines that begin with
‘From ’ and prefix them with a ‘>’. Some mail readers don’t
deal properly with a ‘From ’ line embedded in the body of a
mail message and will break the message into two pieces (I've
gotten reports that
Mozilla Thunderbird
is one of these. I don’t use it, but it would be a shame if
it didn’t work with mailboxes generated by postoffice), so this
option tells postoffice to correct that feature.Another bugfix release. This one corrects a problem that I put in when I released 1.3.pre2 — I'd implemented teergrube detection in runq, but apparently I didn’t test it properly, because when one of the google mailservers decided to really teergrube one of my servers in the middle of this week, Postoffice responded by mutating into a nasty cpu-consuming daemon. Firing up a debugger and attaching to the offending process revealed a spectacular infinite loop of cascading segmentation violations that looked like it would be happy to keep running until timet wrapped and shut the server down. It was a bit of stupid coding; I was closing the connection inside the teergrube detector, and the upper levels of my code didn’t expect that they'd ever have a FILE* deleted out from under them. Thus a spectacular panic loop.
This is not what most people want from their mailservers, so I've taken that feature out.
This release fixes one horrible bug, tweaks the build, and adds one teeny feature. If you're using any of the 1.3 versions, you need to upgrade to this version.
The bug is even more spectacular than the bugs that were in
1.3.1; if a connection fails inside the sendmail filter library,
the library attempts to perror() an error message. This does
not work too well when the smtpd daemon has closed stderr,
because it does not immediately crash but instead staggers
along shedding large chunks of flaming connection debris for
several seconds after the stdio library has written random
garbage all over the session data. The audit logs and coal
database on pell got to be pretty interesting, but not in a
good way, within a day after this bug manifested itself.
The feature is pretty trivial; Postoffice now
tells you its version number when you do a SMTP DEBUG command.
The build tweak is that I now stash the version number in the
auto generated file version.c instead of defining it on the
cc command line and having everyone use the magic define.
One of the features in 1.3.1 (wildcard aliases for secondary mxes) was put in in a hurry because pell had crashed and I wanted to get gehenna set up as a secondary mail server in a hurry (in the old days you'd have 48 to 72 hours before mail started to bounce, but these days you're lucky if you get 12 hours before the big isp start tossing mail into the trash. So I hurried, and ended up with a null pointer dereference) made the SMTP server crash if someone tried to send mail to a nonexistant user in a virtual domain. This is worthy of a fast bugfix release, but I also was doing some work trying to attach more sendmail filters to the copy of postoffice and discovered a potential stack overflow and a place where soft filters (the default right now) would not fail open in the sendmail filter library.
Three bugs, one of which causes crashes, one of which may cause
crashes, and one of which causes incorrect behavior. You
probably want to upgrade to 1.3.2, unless having repeated
CRASH with signal 11 syslog entries is your
idea of a good time.
Version 1.3 is the official release of the new code that supports
sendmail filters (“milters”) using the undocumented wire protocol
that Todd Vierling reverse-engineered for a perl(python?) milter
project. It also includes the trivial change of setting
MAILERCONF up as a make variable, so if you're making
--with-mailwrappers, you can have it write the new mailer.conf
into a different file with make MAILERCONF=path install.
The reason I've rolled postoffice up to 1.3 with such a trivial change from 1.3.pre3 is that I'm not using the milter code on some live machines, and it does not appear to be failing (yet) in production.
* -> foo
would end up expanding * -> * and triggering, correctly,
the alias loop detection code.) This, on top of a few tweaks
to the sendmail milter code (STILL being tested), forces a new
release of the program./path/to/socket) sockets 1.3.pre1 supported, and I've gone
back in and reworked the timeout system in runq so that it’s
a little more robust and will actually time out when some large
free mail providers decide to teergrube my connection. And,
as a shocker, I've actually documented the filter= code in
the manpage for postoffice.cf.Version 1.3.pre1 implements the MTA side of the sendmail “milter”
mail filter protocol so that people can put in AV and spam scanners
without having to hand-code a --with-av. The milter interface
is enabled by the --with-milter flag in configure.sh (replacing
the old --with-av interface). Milters are defined by setting
filter=path-to-socket in /etc/postoffice.cf (only unix-domain
sockets right now; the code to do ipv4 domain sockets is trivially
simple, but I'm not going to put it in until I've had a chance to
run the code for a while.
Postoffice does not yet completely implement the milter protocol ; requests to add, delete, or modify headers and mail recipients are silently ignored, and requests to modify message bodies are treated as a message failure. I am undecided whether to implement these features, so I'm going to leave them out until the code has been beaten upon for a while (I lean towards not implementing those features, since I don’t want my MTA to do any message manipulation beyond the traditional addition of Received: headers.)
You may ask why this is version 1.3.pre1 instead of version 1.3. Well, Pell is running a much-modified SLS distribution, and the a.out libraries I use don’t support pthreads yet. All of the milter libraries and virus scanning daemons out there require pthreads, so I can’t easily test the milter features. If I release the code, there might be more eyes looking at it and finding the little surprises that otherwise could wait for years before popping out and going Hello!
There are also a couple of bugfixes in 1.3.pre1;
Occasionally, a large US ISP/search engine company will
decide that it’s a good day to
teergrube
a mail connection, so runq will hang up forever ; I've
added some additional timeout code to the remote mail
sending routines so that it will pull the plug after 900
seconds no matter what.
FreeBSD and DragonflyBSD put their mailer.conf file in a different place from NetBSD and OpenBSD. I was assuming that the whole world was FreeBSD, and this bit me on NetBSD, so I fixed it.
On Linux 2.4 (and 2.6), the kernel “helpfully” spits out
error messages when you SIG_IGN SIGALRM, then call
wait(); I've tweaked the signal handling to shut this off.
The postoffice spec file (for building rpms) wasn’t being paranoid enough about setting the permissions of /var/spool/mail to 1777, and I'd hardcoded the version#. Both have been cleaned up, and seemed to work on the single RH machine I tried to install the postoffice rpm on.
setproctitle()
for compatability with systems that already have a setproctitle()
of their own. I'd carefully modified configure.sh to check
for setproctitle() so I could conditionally compile my own,
but for some odd reason the changes never got checked in on
the correct machine, and they never made it into 1.2.4. So,
version 1.2.6, which fixes setproctitle() so it won’t blow up
on any of the BSDs out there. Sigh.res_query, and some modern name servers were returning records
short enough to be valid, but not long enough for the resolver
to work) and adds, in the new os/redhat,
an rpm specfile that generates a redhat package for
postoffice.-qXX option into the SMTP daemon. It
turns out that, aside from user requests, it’s easier to build
a r*dh*t linux package when I don’t have to poke around inside
the crontab-format-of-the-day™ to start the queue runner.1.2.1 continues the portability cleanup of the code, by putting in the following features:
More cleanup of the networking code; In mx.c, i was
making many assumptions of how struct in_addr related to
in_addr_t, and those have been replaced with calls to
inet_makeaddr().
On machines that have mailwrappers(8), postoffice
installs itself by writing a mailer.conf instead of deleting
the mailwrapper links and putting binaries down.
And, of course, there are more configure.sh-related tweaks so that postoffice will build on non-intel non-mastodon hardware.
As a bonus, I've started describing how some of the features work.
The antivirus scanner hook --with-av={pgm} assumes that the
av program is a filter that takes a file to scan from stdin, but I
never actually documented this. This can lead to unexpected
surprises when you're not me.
1.2.0 has one moderately large new feature; I've implemented
parts of rfc 2554 to
enable the SMTP AUTH LOGIN command. If configured with
--with-auth, people in virtual domains may log into their
postoffice server and relay mail out to the rest of the world
as if they were actually logged on to the machine. It’s not
very well tested yet, which is why it is only set up for virtual
domains and you have to configure it in instead of simply
turning on a configuration switch, but it’s there and it works
at making a remote user magically into a local user for purposes
of mail forwarding.
In other feature enhancements, I have been informed by reliable sources that, contrary to popular belief, the world is NOT all a VAX. The NetBSD port started in v1.1.11 had a small handful of other stupid machine dependencies that were causing postoffice to either not work correctly or dump core, but I believe that I've managed to weed them out without breaking the rest of the code.
statfs()
function by leaving the function in libc, but removed references
to struct statfs from the header files that the manpage claims
that they can be found in. This is inconvenient, because
configure.sh finds the function call and (foolishly) assumes
that just because NetBSD is a Berkeley Unix it will act like
a Berkeley Unix. To correct this behavior, I've updated
configure.sh to check for both the function and the structure
(since one is of no use without the other) and I've added
support for statvfs(), which appears to be how the kool kids
do statfs() these days.1.1.9 contains quite a few little changes from 1.1.7a, and adds some features to deal with a world full of luserish antispam software.
Many systems are using a nasty antispam “protection”
system that does callbacks to the MX for a SMTP client, and
which attempt to do MAIL FROM:<> as part of this callback.
When they fail to successfully do this callback, they WHINE
AND WHINE AND WHINE about how the system is Violating the
RFCs! and it’s Not Helping Prevent Spam! (I plead guilty
to the first offense, but at least when postoffice is configured
to bounce MAIL FROM:<> it bounces it, instead of accepting
it and then dropping it on the floor like so many sites do
when they receive MAIL TO:<>. But on the second offense,
I emphatically plead innocent because when I replaced sendmail
with postoffice on my public server, people in
virtual domains saw incoming spam drop from 300 messages a
day (of which a vast majority were MAIL FROM:<>)
down to two messages a day.)
However, some of the more aggressive followers of this sort of fake antispam solution have started refusing mail when the callback doesn’t work. Combine this with the latest round of nic obfuscation (ripe has stopped publishing email addresses in their whois information, probably because too many spam houses were getting their mailboxes filled with enraged “TAKE ME OFF YOUR MAILING LIST NOW!” letters, and it means that a large number of spamhauses don’t get the love they deserve after they spam me. So I've redone the nodaemon code so that it doesn’t reject the offending letter until the spam client sends a DATA command, at which point postoffice punts the offending communication.
After a run-in with a particularly luserish systems administrator who WOULD NOT believe that the problem was on his end because postoffice wasn’t spitting out the error code he thought it should be, I've redone the error codes that are returned so that they more closely match the reference implementation.
When spitting out messages in SMTP mode, don’t uppercase text that’s
quoted by < … >.
Authexpire now has the -a option, which is used to instantly
authorize a sender that you don’t want to wait a hour for
the regular old greylist code to approve.
Clean up the code that runs the queue; there were cases where people were repeatedly sent copies of a mail message because the mail was also addressed to someone who was having temporary mail delivery problems.
A whole bunch of internal cleanup to sweep out duplicated and malfunctioning code. The code that handles bounce notifies is still disfunctional, but it’s much less disfunctional now than it used to be.
1.1.7a catches some long-standing bugs and adds a few more features to deal with disk-full denial of service bugs.
There are some systems that don’t seem to properly support
flock(LOCK_EX|LOCK_NB), thus making my spiffy new
flock()-only locking useless on those machines. I've
added in a configuration check to see if flock() actually
works before using it. If it doesn’t work,
postoffice reverts back to the old-style
lockfile locking scheme.
I discovered a potentially nasty denial of service bug when an
antivirus helper program is configured in.
<Postoffice assumes that if the
antivirus program returns a nonzero status, it has found
an error and the message needs to be bounced. If your
disk fills up, mail will mysteriously begin to bounce.
A proper fix would be to put in some way of registering a
“found a virus” return code, but a quick kludge is to check
for a minimum amount of free disk before accepting a message,
and that’s what I put in. If the statfs function is
found, postoffice will not accept mail unless
there is 10mb free on the spool partition (this can be
modified with the minfree option (not documented anywhere
except this page.)
The source of the international **SIGPIPE** conspiracy has
been caught; I was getting occasional crashes for no
apparent reason, and it turned out that if a client went
away too fast postoffice would be killed by
SIGPIPE when it tried to write the “bye bye” message to
the client. I now catch SIGPIPE and ignore it; if the
pipe closes too fast, I'll catch it when I try to read from
the pipe next time around.
There has been some further tuning of the configure.sh
code that looks for ndbm vs gdbm; on some modern
Linux machines, you can find a ndbm
header file which is a decoy file for gdbm. So I had to
redo the code that checks for ndbm and gdbm so that if it
finds ndbm, but doesn’t find the dbm_open() function, it
will then try to use gdbm. I've also added the --use-gdbm
option, which chooses gdbm even if ndbm exists on the
system.
Add the -m (ignored) option for send-mail.
There are still a few outstanding bugs I need to clean up; occasionally, a spool controlfile will go down to zero bytes, and then it will stay there forever until someone goes in and hand-deletes it. And I've realized that I need to spool a control file for local mail delivery, so if postoffice is killed during local mail delivery, it will save the undelivered mail for the next queue run. This is a pretty nasty problem, and the fix needs to go into the next release of the code.
1.1.6 fixes some problems, changes some code, and corrects some bugs;
The Received: header is now generated whenever mail is processed;
it was generated when the mail was being delivered to a
local address, and not when it was to be sent to a remote
machine. Since the whole idea of the Received: header
is to track how the mail flows in the network, this was
less than perfect behavior.
There were a couple of spiffy bugs in queue handling. The bad bug (which is not fixed yet) was that when postoffice tries to send mail to a site that’s really slow, the other connections might time out and disconnect. On my freebsd boxes, postoffice just falls over. This brought up a couple of other bugs;
flock(2) control
files and have the locks release themselves if something
kills a runq?It’s probably not a surprise for me to tell the world that there are functions with side-effects inside postoffice. Well, I'd managed to optimize some code paths so these functions weren’t called all the time they needed to be.
When I added support for gdbm databases, I broke support for
ndbm databases. Since all of my machines use ndbm,
this was pretty quickly discovered after I moved the 1.1.5
database portability layer off the R*dh*t box I was developing
it on.
1.1.5 continues along the slow path of documenting everything, it cleans up a trivial bug that makes runq dump core if a SMTP server allowed postoffice to connect, but then disconnected with saying anything, and adds some more features, like
Support for gdbm-style databases on
Linux systems that don’t ship with
ndbm (by simply abstracting the database layer out;
the database interface code does both ndbm and gdbm, and
could easily be modified to support the fly swatting
battleship that is sleepycat db4.) Note that just because
this allows compilation on modern Linux systems, this
doesn’t mean it’s a drop-in replacement for sendmail
— you need to newalias after installation, because
gdbm is not compatable with db3.
Postoffice now configures and builds on Redhat Linux 8.0.
I've added new options so that postoffice
can operate in a network with a private dns; the
forward-all option, if relay-host is set, tells
postoffice to forward mail addressed to
unknown hosts to the relay-host for processing.
I've documented the ndbm program!
README file describing the program (my writing skills have
managed to completely evaporate in the last month, so it’s not
a very impressive README), cleans up some of the internal
logging code, and modifies, slightly, the behavior of MAIL FROM:.
The change in MAIL FROM: is to turn off a lot of the
paranoid domain debugging (even when verify-from is set) when
you give an address from one of the virtual domains for the
system, because since someone from a virtual domain doesn’t
actually have an account, the only way their domain will show
up will via a connection from a remote site.1.1.3 fixes quite a few bugs, works around some bugs in broken third-party software, and adds in support for vm-pop3d-style virtual domains. The virtual domains are the reason for the release; I donate web and mail hosting on my public server, and I used to use sendmail, vm-pop3d, and some homebuild glue programs to pick up and deliver mail to people in some of these domains, but some of the users were getting upwards of 500 virus, pieces of spam, and chain letters a day, so I had to either spend more time than I wanted to upgrading, patching, and wedging antispam into sendmail or improving <Postoffice to handle virtual domains.
Improving Postoffice was the obvious choice. A
few buglets were discovered during the coding; I'd miscoded a
loop during dns resolution which was causing additional dns
query traffic, I was incorrectly setting NOBODY_GID to its
user ID instead of the group ID, and I wasn’t actually using
NOBODY_UID and NOBODY_GID anywhere where I should have been.
There are some featurettes in 1.1.3 too:
verify-from, which relaxes paranoia
checking on MAIL FROM: arguments.Orc@postoffice,
postoffice was properly realizing that the mail
is going to orc@postoffice, but then got confused about the
mailbox and wrote the mail to /var/spool/Orc instead of the
correct one. And I don’t even run Microsoft Windows, so I'm
not on a filesystem that would hide that behavior. But it
should be fixed now, and hopefully in a way that won’t surprise
me in the near future.This release adds a new feature; I've added enough options so
that it’s cumbersome to pass them all on the command line, plus
if a user runs some programs that depend on the options (runq,
for example, depends on relay-host) they can’t set some of
those options and thus postoffice will relay mail
differently. The file /etc/postoffice.cf contains options
(one per line, no leading -o, and you can have # comments)
that will always be read in, even if a program is being run by
a nonpriviledged user. There’s also a little bug fix;
4xx
banner. The SMTP spec says that 4xx means that the server
will spit out the banner, then drop the connection; since
we weren’t handling 4xx properly, postoffice
would read the banner then attempt to write to the (now
closed) tcp connection, with hilarious results.sendmail -t, udp
instead of tcp dns lookups, and a slightly simpler mail
format for bounce messages.I had to put in a tweak to the greylisting code over the weekend;
some mail servers (verizon.net is an example) do callbacks when
you attempt to send mail to them and if the
MAIL FROM:<…> is greylisted, it will promptly
refuse your mail. So I've tweaked the code a little bit so
that even if you're greylisted, the MAIL FROM: will
return a 2xx reply, but the DATA command will fail with
a 4xx error.
I've also implemented getif for FreeBSD
by the simple and kind of ugly hack of calling /sbin/ifconfig
(FreeBSD uses much the same sort of approach as Linux does for
getting back a list of configured interfaces, so I can do it
programmatically, but it’s even uglier than the already hideously
ugly Linux approach) and parsing out the interface addresses.
A honest-to-got bug has been found and removed, too; runq was delivering bounce messages when it was unable to connect to a remote mail server. It wasn’t bouncing the mail, no, it was doing a “could not deliver mail to: Reason: connection refused.”. This was not a particularly useful sort of message to clutter up my mailbox with.
coal handling to make it
harder to be given coal, I've put in the -orelay-host= option
that makes Postoffice send all remote mail through
a relay host.A few more days of running the code found some buffer overruns, and then building the code on a FreeBSD box found some Linux dependencies, which have been swept out, no doubt to be replaced with some still to be discovered bugs.
--with-coal implements an experimental firewall list,
which calls ipfw to blacklist sites if they send too much
bad traffic into the server. This is really experimental,
and may drop out of the next version if I don’t like the
way it firewalls off systems.
The -a flag turns on session auditing, which syslogs every command from SMTP clients.
The -ocheckhelo option tells Postoffice
to refuse connections from clients to claim to be us
when they say HELO or EHLO.
Now it’s been updated so that it uses configure.sh instead of having to hand-edit configuration files whenever I to build it on platforms other than pell. I also found and got rid of a horrible programming mistake where I was attempting to directly use scratch buffers when doing remote mailing (this had the nasty side-effect of occasionally misdelivering mail). It also bounces messages when they've sat in the queue too long, and I've got many manpages written for all of the cruft I'm busily welding to the superstructure.
Surprisingly, it still picks up and delivers mail.
sendmail -t.