Monday, 25 September 2006

SSH as a SOCKS proxy revisited

A while ago I had tried to use SSH as a web proxy and didn't have any success in getting it to actually work. I was worried that maybe this was a NAT-to-NAT complication or something of the sort but no, it was just a client configuration problem. It turns out the trick is to just set the SOCKS settings in Firefox rather then the HTTP proxy. Firefox will choose the HTTP proxy first or maybe I just don't understand the difference, but here's how it works for me:

ssh -D port host

e.g.
ssh -D 3125 siona.nibble.bz


Then configure Firefox (Edit - Preferences - Connection Settings) with a SOCKS proxy of localhost, port 3125 like this:



The best test is to go to What Is My IP? and refresh the page with the proxy disabled/enabled and verify the IP address changes.

Now for shits and giggles, you use -f and -N with SSH to background the ssh process (the -f) without running any remote command (the -N) like this:

ssh -fN -D 3125 siona.nibble.bz


This will leave your SOCKS proxy in the background so you can close your terminal and still surf through the proxy.

Hooray for bypassing crappy firewalls and HTTP proxies!

History Meme

14:30:42 % history 1|awk '{print $2}'|awk 'BEGIN {FS="|"} {print $1}'|sort|uniq -c|sort -nr|head -10
394 ssh
173 ls
138 cd
93 gpg
60 sudo
58 host
56 vim
48 killall
43 date
39 man


Apparently I like to shell around killing programs. Exciting? Maybe not...

Updates for SASL

The latest updates for Siona included some new SASL packages. These stomped some of the totally garbled config items from my setup for SMTP-AUTH but in the end, there was only one change I had to make: add postfix user to the sasl group. That was it. Auth started working again as soon as I kicked Postfix.

Woo!

Monday, 18 September 2006

SMTP-AUTH With Postfix

So I finally got authenticated SMTP working. So in addition to accepting mail for domains for which Siona is a relay and relaying mail from the local network, authenticated remote or mobile users can now relay mail as well. My criteria for this solution were that a) the authentication would be compatible/integrated with PAM, b) secure, and c) not some hokey POP-before-SMTP setup. So the solution is to have postfix authenticate against Cyrus saslauthd and then saslauthd authenticate with PAM (and PAM with LDAP, but that's just details).

Siona is running Debian Sarge with Postfix 2.2. So it turns out that in addition to the installed postfix package, I need sasl2-bin which includes the essential saslauthd and the useful testsaslauthd.

After installing saslauthd, it must first be enabled. The default file /etc/default/saslauthd should read:

START=yes
PARAMS="-m /var/spool/postfix/var/run/saslauthd"
MECHANISMS="pam"


You will need to create the directory listed above. A simple "mkdir -p" will do the trick.

Note: One page on the Internet recommended a PARAMS line which included a "-r". This will not work and cause all auth attempts to mysteriously fail since it is attempting to combine the local part and realm.

Note 2: Online I also so reference that you should change the saslauthd init script to move the PID into Postfix's chroot. This is also a lie. It won't make anything fail, it's just not needed.

So that's it for saslauthd. If you run testsaslauthd like this, you should get an "OK" for any existing user account:

$ testsaslauthd -u username -p password -s /var/spool/postfix/var/run/saslauthd/mux
:0 OK

Now onto postfix. First, let's finish up the SASL parameters. In /etc/postfix, there should be a sasl folder. If not create it. In there you create smtpd.conf which will contain the parameters for smtpd to actually use saslauthd and actually there's just two lines:


pwcheck_method: saslauthd
mech_list: plain


The first line is, you guessed it, the authentication method which in our case is the saslauthd. The second line just says which authentication methods are supported. For fancy (e.g. secure) methods like CRAM-MD5 and such, you need a separate password database. We don't have that so we're limited to the standard PLAIN and Microsoft's LOGIN. Since I don't need to support old Microsoft clients (though others do so watch out for this), I only enable PLAIN. Other fun types include OTP which provies a fancy one-time-password exchange. Very neat, but that's for another day.

The one thing to keep in mind is that this is a config file for Postfix, not saslauthd, so if you make changes to this file, it's Postfix that has to be restarted.

Note 3: Those are the only two lines you need. Anything else, like the socket path, is frivolous with our setup.

Note 4: There are standard locations to put the smtpd.conf, like /usr/lib/sasl2 in Debian, or /usr/local/lib/sasl2 when SASL is built with default paths. Some of these will work and others won't. In my case, the /etc/postfix/sasl worked as expected.

Now that we've written/uncommented a whopping 4 lines, let's get into Postfix's main.cf and do the last 4 lines. In main.cf, we want to a) enable sasl, b) permit sasl authenticated senders, and c) force TLS since our only auth method is "PLAIN".

In main.cf, add the following lines:

smtpd_sasl_local_domain = $mydomain
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination,
check_recipient_access hash:/etc/postfix/role_exceptions

smtpd_sasl_authenticated_header = yes
smtpd_tls_auth_only = yes


For your sasl authentication, if your site is setup anywhere near sanely, then your domain is going to be your realm. Since we are doing PAM authentication, I'm not even sure this is required. I haven't tested it though so I've left it in.

The following line enables SASL, big surprise there ;)

In the smtpd_recipient_restrcitions, I have added the line permitting SASL authenticated connections. Make sure this line comes after your "mynetworks" line. Failing SASL is going to get your connection booted if you order it higher (last time I tested, not recently, I admit).

The SASL authenticated header, I'm not exactly clear what this does but it is "no" by default and most online documentation says you need to set it to "yes".

Lastly, the TLS auth only line requires TLS before AUTH is permitted. This will force all your users to enable TLS to protect their passwords (and subsequently their messages) in transit.

And now just kick postfix and you've got authenticated SMTP enabled for remote clients! You may have noticed that I have many rambling notes in my post. As you may infer, I got pretty frustrated in the end with instructions that were ambiguous, useless, or just plain wrong. Some of the stuff, like the -r for saslauthd, I don't know how the user even got their system to work... Ah well, c'est la vie.

Early Move

Although I had planned to move at the end of the month, my Internet service went out last week so I have moved siona a bit ahead of schedule. The Internet is still out so I moved siona to a friend's place (hooray) so I got to go through all the pains that go along with changing the IP address of the primary name server. Hooray. We're still trying to convince our secondary dns service to figure out what's going on. It's being a bit stubborn.

Nevertheless, everything has been moved and everything will have to be moved again once I move and move siona to the new house. Definitely looking forward to it like I look forward to getting hit in the face.

Thursday, 31 August 2006

Bring on the Pr0n

A little while ago I was thinking about transcoding a bunch of my music from Ogg to Mp3 so I could burn the music to CD for my Mp3 CD player. Now I'm no stickler for audio quality, but clearly transcoding from one lossy format to another is going to make things worse then necessary. Given I still have some desire to have music available both streamed and on Mp3 CD, I decided to start re-encoding my music collection to a lossless audio format, specifically FLAC.

Ripping to FLAC was going well initially but there is one distinct downside to using a lossless format which is the disk space requirements. My music collection will be jumping from ~15GB up to 100GB. After some messing around with my volumes on Siona, I could not come up with enough free space without destroying a lot of data. Well, I destroyed a bunch of data for good measure (and because it is important to test that you can restore from backups) and then I bought a new drive.

So the new drive is a spanking new 320GB Seagate drive. I slapped that bad-boy in there and formatted it with XFS. So far so good. I've moved all the existing data from the old WD 80GB drive on there and I've been ripping CDs merrily.

To assist with the task, I even created a little shell script that gives me a list of every CD already on the system and whether the CD is FLAC or not. Even lumps the Various Artist discs together. I'm almost up to a third of the CD collection (between both the wendigo's and mine).

Friday, 25 August 2006

The Fight Against Spam

The previous post was about my experience dabbling with greylisting but what I really rely on for mail filtering is Amavis, SpamAssassin, ClamAV, and Procmail. Each tool has it's place and does it's job quite well. Except Procmail, but that's just a backwards old mail processing/delivery agent that using syntax far too arcane for mere mortals... But that's just my griping for no good reason, it does the job.

Anyhow, the setup here is that Postfix hands all messages to Amavis for inspection. Amavis can run a message through any number of spam or virus checking programs, SpamAssassin and ClamAV in our case, and any of these programs can approve a message, mark it in some way, move it to a quarantine or reject it flat-out. Basically, Amavis is a front-end for these types of filtering applications. It's very versatile, postfix works well with it, so it works.

The simple one is ClamAV. It runs a message through its virus definitions. If it finds a match, it quarantines the message and then sends a notice to the user saying what happened. Easy. ClamAV actually picks up much of the Phishing scams. It works great.

The other fun one is SpamAssassin (SA) which reads through a message and assigns it a score depending on many factors like whether the message headers appear corrupt. Low score means probably not spam, high score means probably spam. If it's a high score, SA can modify the message or discard it. The levels at which it takes "evasive action" are configurable so this has taken some tuning before I was really happy with the results.

So the setup here is that SA actually modifies all messages adding a X-Spam-Score: nn header to all messages. This way, I can see that the last message I got from my roommate was scored -3.787, for example.

At a score of 4.0, SA adds an additional header that reads X-Spam: Yes and also adds ***SPAM*** to the subject line of the message. This is where Procmail comes in. I have Procmail configured on my account to automatically move any messages with X-Spam: Yes to my Junk folder and out of my inbox. I have found some false positives, specifically my logwatch notices from Siona, which can sometimes go above 4.0 so SA is configured to still deliver messages to the user.

Above 6.0, however, I have seen no false-positives so SA is configured to reject these messages. Above 10.0, SA will not even bother sending a delivery status notification (DSN) to the source of the email on the assumption that the source email address is spoofed and the sending mail server is just a zombie host.

This has been working out very well. My guess is that 90% of spam is scored over 6.0 and is outright discarded. Of the remaining 10% of spam, I would guess 80-90% of that is scored above 4.0 and is getting tagged as spam but delivered. A very small number of legit messages are scored between 4.0 and 6.0 (less then 1%), and no legit messages are scored above 6.0. All in all, this leaves a couple percent of the spam messages that are delivered to the users.

This squad of applications works quite well together for fighting spam. I only have a couple of problems. The setup is a bit confusing out of the gate, but definitely doable and there was a lot of tutorials Online for this setup. And the other problem is that we should not be seeing this much spam anyhow. Mail servers should be authenticating users that wish to relay and message authenticity should be verifiable. If Joe Smith at example.com gets a message from my domain, the example.com mail server should be able to verify that someone at this domain actually sent the message. As it is, it is trivial to forge a source email address, and SPF is not the solution. We need something universal, unlike SenderID, or else what's the point? The closest is the domain signing that Yahoo (?) came up with (which Google actually uses for their mail service), but it is still not universal. Someday, we will leave with not much spam, some day...

Popular Posts