Dec 6: User authentication

This blog post is part of the FastMail 2014 Advent Calendar.

The previous post on December 5th was about the importance of data integrity. The following post on December 7th is about how we install servers.

Technical level: medium/high

Today we talk about the internals of our authentication system, which is how we decide that you are who you say you are and from there, figure out what you’re allowed to do.

On every server we have a service running called “saslperld”. As with many of our internal services, its name is now only loosely related to its function. saslperld is our authentication service, and exists to answer the question “can this user, with this password, access this service?”.

Making an authentication request

Each piece of server software we have has some way of calling out to an external service for authentication information. Each server tends to implement its own protocol for doing this, so saslperld implements a range of different methods to receive authentication questions.

The simplest of these is the original saslauthd interface, used by Cyrus. It has three fields – username, password, and service name, and returns a simple yes/no answer. These days it’s barely used, and only really in internal services, because it can’t really be extended so we can’t pass in other interesting information about the authentication attempt (which I’ll talk about further down).

The real workhorse is the HTTP interface, used by nginx. Briefly, nginx is our frontend server through which all user web and mail connections go. It takes care of authentication before anything ever touches a backend server. Since it’s fundamentally a system for web requests, its callouts are all done over HTTP. That’s useful, because it means we can demonstrate an authentication handshake with simple command-line tools.

Here’s a successful authentication exchange to saslperld, using curl:

$ curl -i -H 'Auth-User: robn@fastmail.fm' -H 'Auth-Pass: ********' -H 'Auth-Protocol: imap' http://localhost:7777
HTTP/1.0 200 OK
Auth-Pass: ********
Auth-Port: 2143
Auth-Server: 10.202.80.1
Auth-Status: OK
Auth-User: robn@fastmail.fm

The response here is interesting. Because we can use any HTTP headers we like in the request and the response, we can return other useful information. In this case, `Auth-Status: OK` is the “yes, the user may login” response. The other stuff helps nginx decide where to proxy the user’s connection to. Auth-Server and Auth-Port are the location of the backend Cyrus server where my mail is currently stored. In a failover situation, these can be different. Auth-User and Auth-Pass are a username and password that will work for the login to the backend server. Auth-User will usually, but not always, be the same as the username that was logged in with (it can be different on a couple of domains that allow login with aliases). Armed with this information, nginx can set up the connection and then let the mail flow.

The failure case has a much simpler result:

HTTP/1.0 200 OK
Auth-Status: Incorrect username or password.
Auth-Wait: 3

Any status that isn’t “OK” is an error string to return to the user (if possible, not all protocols have a way to report this). Auth-Wait is a number of seconds to pause the connection before returning a result. That’s a throttling mechanism to help protect against password brute-forcing attacks.

If the user’s backend is currently down, we return:

HTTP/1.0 200 OK
Auth-Status: WAIT
Auth-Wait: 1

This tells nginx to wait one second (“Auth-Wait: 1″) and the retry the authentication. This allows the blocking saslperld daemon to answer other requests, while not returning a response to the user. This is what we use when doing a controlled failover between backend servers, so there is no user-visible downtime even though we shut down the first backend and then force replication to complete before allowing access to the other backend.

This is a simple example. In reality we pass some additional headers in, including the remote IP address, whether the conneciton is on an SSL port or not, and so on. This information contributes to the authentication result. For example, if we’ve blocked a particular IP, then we will always return “auth failed”, even if the user could have logged in otherwise. There’s a lot of flexibility in this. We also do some rate tracking and limiting based on the IP address, to protect against misbehaving clients and other things. This is all handled by another service called “ratetrack” (finally, something named correctly!) which all saslperlds communicate with. We won’t talk about that any more today.

There’s a couple of other methods available to make an authentication request, but they’re quite specialised and not as powerful as the HTTP method. We won’t talk about those because they’re really not that interesting.

Checking the password

Once saslperld has received an authentication request, it first has to make sure that the correct password has been provided for the given username. That should be simple, but we have our alternate login system that can make this quite involved.

The first test is the simplest – make sure the user exists! If it doesn’t, obviously authentication fails.

Next, we check the provided password against the user’s “master” password. There’s nothing unusual here, it’s just a regular password compare (we use the bcrypt function for our passwords). If it succeeds, which it does for most users that only have a single master password set, then the authencation succeeds.

If it fails, we look at the alternate logins the user has configured. For each one available, we do the appropriate work for its type. For most of these the provided password is a base password plus some two-factor token. We check the base password, and then perform the appropriate check against the token, for example a Yubikey server check, or comparing against the list of generated one-time-passwords, or so on. The SMS 1-hour token is particularly interesting – we add a code to our database, SMS the code to the user, and then fail the authentication. When the user then uses the code, we do a complete one-time-password check.

At this point if any of the user’s authentication methods have succeeded, we can move on to the next step. Otherwise, authentication has failed, and we report this back to the requesting service and it does whatever it does to report that to the user.

Authorising the user

At this point we’ve verified that the user is who they say they are. Now we to find out if they’re allowed to have access to the service they asked for.

First, we do some basic sanity checking on the request. For example, if you’ve tried to do an IMAP login to something other than mail.messagingengine.com, or you try to do a non-SSL login to something that isn’t on the “insecure” service, then we’ll refuse the login with an appropriate error. These don’t tend to happen very often now that we have separate service IPs for most things, but the code is still there.

Next, we check if the user is allowed to login to the given service. Each user has a set of flags indicating which services they’re allowed to login to. We can set these flags on a case-by-case basis, usually in response to some support issue. If they user is not explicitly blocked in this way, we then check their service level to see if the requested service is allowed at that service level. A great example here is CalDAV, which is not available to Lite accounts. An attempt to by a Lite user to do a CalDAV login will fail at this point. Finally, we make sure that the service is allowed according to the login type. This is how “restricted” logins are implemented – there’s a list of “allowed” services for restricted accounts, and the requested service has to be in that list.

(It’s here that we also send the “you’re not allowed to use that service” email).

Once we’ve confirmed that the user is allowed to access the requested service we do a rate limit check as mentioned above. If that passes, the user is allowed in, and we return success to the requesting service.

The rest

To make things fast, we cache user information and results quite aggressively, so we can decide what to do very quickly. The downside of this is that if you change your password, add an alternate login method or upgrade your account all the saslperlds need to know this and refresh their knowledge of you. This is done by the service that made the change (usually a web server or some internal tool) by sending a network broadcast with the username in it. saslperld is listening for this broadcast and drops its cache when it receives it.

There’s not much else to say. It’s a single piece of our infrastructure that does a single task very well, with a lot of flexibility and power built in. We have a few other services in our infrastructure that could be described similarly. We’ll be writing more about some of those this month.

Posted in Uncategorized. Comments Off

FastMail’s servers are in the US: what this means for you

As you may be aware, there have been many reports recently of secret data mining programs conducted by the US government. These reports have included mention of secret network interception and “backdoor” access to private email accounts. While we have no position on the veracity of these claims, we have had many queries about what, if anything, this might mean for us and for our customers, given that our primary servers are located in the US. This is our response to those questions.

As noted in our recently updated privacy policy, we are an Australian company subject to Australian law. We are required to disclose information about specific individual accounts to properly authorised Australian law enforcement with the appropriate supporting documentation, which means a warrant signed by an Australian judge. We do not co-operate with any kind of blanket surveillance, monitoring or “fishing expeditions”, and we do not give out user information to anyone outside Australia.

There are some rare cases of overseas entities applying to Australian authorities for information under a Mutual Assistance treaty. In that case proper Australian documentation must be issued before we will do anything. These cases are particularly rare because the burden of proof required is very high, and the chain of events is very long (ultimately, each case currently requires sign off from the Australian Attorney General). The Mutual Assistance treaties also have (amongst other things) a test of whether the putative offense would be illegal in Australia, not just in their country of origin.

Australia does not have any equivalent to the US National Security Letter, so we cannot be forced to do something without being allowed to disclose it.

It has been pointed out to us that since we have our servers in the US, we are under US jurisdiction. We do not believe this to be the case. We do not have a legal presence in the US, no company incorporated in the US, no staff in the US, and no one in the US with login access to any servers located in the US. Even if a US court were to serve us with a court order, subpoena or other instruction to hand over user data, Australian communications and privacy law explicitly forbids us from doing so.

It might be possible for the US government to lean on the Australian government or other international legal body to compel us to hand over data but this likely to be an expensive, time-consuming and highly visible process. In our opinion those barriers make it extremely unlikely to happen.

There are of course other avenues available to obtain your data. Our colocation providers could be compelled to give physical access to our servers. Network capturing devices could be installed. And in the worst case an attacker could simply force their way into the datacentre and physically remove our servers.

These are not things we can protect against directly but again, we can make it extremely difficult for these things to occur by using strong encryption and careful systems monitoring. Were anything like this ever to happen we would be talking about it very publically. Such an action would not remain secret for long.

Ultimately though, our opinion is that these kinds of attacks are no different to any other hacking attempt. We can and will do everything in our power to make getting unauthorised access to your data as difficult and expensive as possible, but no online service provider can guarantee that it will never happen.

As a customer, you need to weigh the benefits of keeping your data with us against the risks of that data being disclosed. You may wish to obtain private legal advice to help you assess that risk. And if you come to the conclusion that keeping your data with us is too risky, then we respect that (though please do tell us why!)

If you have any further privacy concerns we haven’t addressed, please email privacy@fastmail.fm.

Posted in Uncategorized. Comments Off

Google Authenticator now supported for two-factor authentication

FastMail has long supported various methods of two-factor authentication for additional account security, from generated one-time-passwords, to SMS, to Yubikey. Today we’ve added another method to our stable – the Google Authenticator method, otherwise known as Time-based One Time Passwords (RFC 6238). With this you can use your iOS, Android or almost any other mobile device as your second factor when authenticating, increasing the security on your account without requiring you to carry an additional object around with you.

A Google Authenticator alternative login can be configured in the Alternate Logins section of your account settings screen. If you’re using the official Google clients, then you can use its support for QR codes to make setup super-easy. You can however choose to use any number of other clients that support this authentication mechanism; all will work with our implementation.

Please refer to our Google Authenticator help page for more details.

Posted in Uncategorized. Comments Off

Changes to webmail login

TL;DR: We’re now making all connections to the Fastmail web interface immediately redirect to a secure (https) connection.

As part of our commitment to making all connections between users computers and our servers secure and encrypted, we’ve just made some changes to our webmail login page. In most cases, users won’t notice any change because we made Secure Login the default almost a year ago. The new changes will only affect the small number of users that have special login requirements.

The main change we’re making is that where previously we would redirect from an insecure (http) to secure (https) connection during login, or on returning to Fastmail on a computer you’d logged in via before, we will now redirect to the secure login screen immediately when you connect to Fastmail. That is, as soon as you go to http://www.fastmail.fm (insecure) or http://www.sent.com (insecure), we’ll always redirect to https://www.fastmail.fm (secure).

Going to other https:// domains that aren’t supported (e.g. https://www.sent.com, a secure connection, but will report a certificate error) will redirect to https://www.fastmail.fm as well.

This will also be the case for businesses and families that use their own domain for logging in (e.g. http://mail.digitalintegrity.com), they’ll also be redirected to https://www.fastmail.fm, but we will continue to correctly show the family/business login screen.

There are a couple of additional exceptions to this.

The mobile UI domains that start with the http://m. prefix like http://m.fastmail.fm (insecure) and http://m.sent.com (insecure) will redirect to https://m.fastmail.fm (secure). This will always show the mobile login screen and mobile interface when you login.

The special "sticky ssl" domains that start with the https://ssl. prefix like https://ssl.fastmail.fm (secure) and https://ssl.sent.com (secure, but certificate warning) will "stick" to that domain. This may be useful as a work around for some proxies that block hostnames with the word "mail" in them.

If for some reason you need to use an insecure login (which we highly recommend you do not do), you will explicitly need to go to the URL http://insecure.fastmail.fm. If you use this to login, data sent between your computer and our server will travel unencrypted over the Internet. This service is only provided for dire circumstances, is highly discouraged, and may be removed in the future.

For the curious, here’s a list of all the transitions that should happen. The "(W)" means you’ll see a certificate warning about mismatched hostnames.

https://www.fastmail.fm               -> stays at https://www.fastmail.fm
http://fastmail.fm                    -> https://www.fastmail.fm
http://sent.com                       -> https://www.fastmail.fm/?domain=sent.com
http://www.fastmail.fm                -> https://www.fastmail.fm
http://www.sent.com                   -> https://www.fastmail.fm/?domain=sent.com
https://fastmail.fm                   -> https://www.fastmail.fm
https://sent.com (W)                  -> https://www.fastmail.fm/?domain=sent.com

http://mail.digitalintegrity.com      -> https://www.fastmail.fm/?domain=digitalintegrity.com
https://mail.digitalintegrity.com (W) -> https://www.fastmail.fm/?domain=digitalintegrity.com

http://m.fastmail.fm                  -> https://m.fastmail.fm
http://m.sent.com                     -> https://m.fastmail.fm/?domain=sent.com
https://m.fastmail.fm                 -> stays at https://m.fastmail.fm
https://m.sent.com (W)                -> https://m.fastmail.fm/?domain=sent.com

http://ssl.fastmail.fm                -> https://ssl.fastmail.fm
http://ssl.sent.com                   -> https://ssl.sent.com/ (W)
https://ssl.fastmail.fm               -> stays at https://ssl.fastmail.fm
https://ssl.sent.com (W)              -> stays at https://ssl.sent.com/ (W)

http://insecure.fastmail.fm           -> stays at http://insecure.fastmail.fm
http://insecure.sent.com              -> stays at http://insecure.sent.com
Posted in News, Uncategorized. Comments Off

Proxy servers for alternate IMAP port (may be useful for Blackberry BIS users)

I recently posted about our new IMAP alternate namespace port that may be helpful for some users.

Some blackberry BIS users reported that their service provider wouldn’t let them change the IMAP port number, which meant that couldn’t use the alternate namespace port. To deal with this, I’ve setup a “proxy” server for the IMAP alternate namespace port.

Our “proxy” servers are special hostnames that listen on every port and respond with a particular service. Our current proxy servers are:

  • imap.proxy.messagingengine.com – IMAP
  • imaps.proxy.messagingengine.com – IMAP SSL
  • pop.proxy.messagingengine.com – POP
  • pops.proxy.messagingengine.com – POP SSL
  • smtp.proxy.messagingengine.com – SMTP
  • smtps.proxy.messagingengine.com – SMTP SSL
  • ldap.proxy.messagingengine.com – LDAP
  • ldaps.proxy.messagingengine.com – LDAP SSL
  • imapalt.proxy.messagingengine.com – IMAP (alternate namespace)
  • imapalts.proxy.messagingengine.com – IMAP (alternate namespace) SSL

So for BIS users, what you would do is:

  1. Login to the BIS website your provider gave you details for
  2. Setup an external email account
  3. When it asks for the username and password, just enter the username and leave the password blank
  4. It should then take you to a page with more advanced configuration options. There you can say to use IMAP + SSL, and when it asks for the servername, use “imapalts.proxy.messagingengine.com”
Posted in Uncategorized. Comments Off
Follow

Get every new post delivered to your Inbox.

Join 5,843 other followers