I'm setting up a high capacity pop server to replace one
thats thrashing it's self to death at the moment, so I've
been trying to find the most efficent pop server for what I
want to do.
At the moment we are running qpopper,
but if a mailbox gets above 50Mb or so the server really
slows down as qpopper copies the entire mailbox
before and after a user transfers their mail.
We need a system that works well with:
- Large (50Mb+) mailboxes
- Many users accessing large mailboxes at the same
time
To get around the problem of large mailboxes I want a pop
server that uses maildir, and for vairous reasons I
want one that runs standalone, i.e. not through
inetd.
After some searching through vairous mailing list
archives and freshmeat
I decided upon Solid
POP3.
The problem now is that the wire-format for POP3 has
all line endings as \r\n, where as the normal unix line
endings are just \n. This means you have to scan through
each message to get a count of the lines so you can account
for transmitting all the line endings as \r\n - you need to
know the size of each message so you can give the right
responce to the LIST command.
With the test mailbox I was using (an excessive 1Gb in
size and 329 messages) this takes 2 minutes!, which is
WAY to long as far as user interactivity goes!
Now, since the only processes that are going to be
touching these mailboxes are the MTA i'm using (exim), and the pop server I
could do whatever I wanted with the mailbox format.
I set up exim to deliver all the mails with \r\n line
endings, and then created a new mailbox type in the solid
pop3 source, crlf_maildir.
Since we don't need to munge the line endings any more
all we have to do for tha vairous pop commands is:
- Getting the data for LIST is just a question of looping
through readdir and stat'ing each file
for it's size.
- RETR is just a mmap and write.
- TOP is just mmap, loop through counting line
endings, and then write. (fetchmail uses TOP
<msg> 99999999, and so might other pop clients, so if
we check to see if no. of lines >= size of message we can
just chuck out the message as a whole...)
- ... and everything else is as before
I've implemented most of this, and it now takes less than
a second to authenticate a client, rather than 2 minute,
which is a nice imporvement ;-)
Expect to see a new version of Solid POP3 soon!