Dec 22: CardDAV beta release

This blog post is part of the FastMail 2014 Advent Calendar.

The previous post on 21st December was about our file storage system. Stay tuned for another post tomorrow.

Technical level: medium

After more than a year of anticipation we’re very happy to announce today that we’re releasing CardDAV support into public beta test.

CardDAV is a protocol for reading, writing and synchronising contact data. It’s built into iOS devices and available on Android with an inexpensive third-party application. If you’ve ever wanted to have your FastMail contacts available on your mobile device (and vice-versa), then this is exactly what you want.

Obviously, since this is a beta, there are still a few pointy edges and non-working bits. The most notable thing is that the beta is currently only available to personal accounts, not to business or family accounts. This is because support for shared contacts is not ready yet and there’s some potential for data loss and inconsistent behaviour if you try to use shared contacts without proper support. We’re working hard on finishing shared contact support and hope to make the beta available to business and family accounts within the next couple of months.

CardDAV is only available to Full account levels and higher. Member, Guest and Lite accounts will need to upgrade to be able to use CardDAV.

So now all the disclaimers are out of the way, you can sign up for the CardDAV beta here: https://www.fastmail.com/go/carddavbeta.

The contacts story

An address book is a fundamental component of any mail system and FastMail has had one almost since the beginning. It’s always been stored in the MySQL database and available through the web client. For much of its history it’s been confined to the web client. A few years back we did add a read-only LDAP interface, which is useful for desktop mail clients that could support LDAP address books. It works fine, but being read-only severely limits its usefulness. Some time later mobile devices happened, and it became clear that something else was needed.

In 2011 the CardDAV protocol was published, largely developed at Apple to allow device contacts to be synchronised with a server. The protocol is very similar to the earlier CalDAV protocol (which we also use for our calendar) which is good as it allows us to share a lot of code between our calendar and contacts system.

Towards the end of 2012 we started to seriously appreciate the need for both an integrated calendar and device contacts syncing. We weren’t the only ones, as the Cyrus project had started to add support for CalDAV and CardDAV to the Cyrus mail server. We looked at a few options for calendar and contacts and decided that we would implement both on top of the support being baked into Cyrus, and work began in earnest. We decided that calendar was more important because we already had a contacts system and while it wasn’t perfect we preferred to focus our engineering effort on a clear gap in our product lineup. That work took the best part of a year, and we finally released the calendar to production in June 2014. At this point we were able to focus on CardDAV-based contacts.

The actual CardDAV part of this work is actually fairly simple. Unlike CalDAV, the backend server (Cyrus) doesn’t really need much special knowledge. It mostly just saves and loads contact cards as required. Calendar entries are more complicated; the server needs to know about timezones, recurring events, alarms, etc. CardDAV is much easier and if all we had to do was ship CardDAV support, we probably could have done so months ago.

The thing that made it more difficult came from the fact that we already had a contacts system and plenty of code fairly tightly integrated with it. It’s more than just the two user interfaces. The mail delivery pipeline also makes use of user contacts for spam whitelists and distribution lists, so we needed to teach these systems about a whole new storage system for contacts. Up until this time they had simply hit the database for this information. To make matters worse, we always knew that we’d need to roll out CardDAV to users gradually which meant that both the UI and the delivery code needed to be able to work with either. In short, we needed to abstract away the implementation details of the contacts storage, which took a few months to build, test and deploy. We ended up with a nice abstraction based on the JMAP getContacts/setContacts model, with a database provider behind it.

The next step was to write a CardDAV provider for our contacts abstraction. That was actually pretty easy because most of the code needed to access DAV resources was already available from our CalDAV work.

The last piece of the puzzle was the actual data conversion layer. The existing contacts system has a data model that doesn’t match up perfectly with the vCard format used by CardDAV, so we had to develop a mapping. Most of the fields have a 1:1 mapping (addresses, email addresses and phone numbers). What we call “online” fields, however, do not. Our “online” field group includes URLs, Twitter handles and chat IDs. vCard doesn’t group those the same way but more annoyingly, it doesn’t have a standard set of fields for representing these. It took a long time to develop and test a mapping that works most of the time. It’s going to need improvement as we go but it’s not bad for now.

What’s next

The next few months will include a lot more testing, polishing and responding to user feedback and obviously completing the business and family support. That will bring us to a full release where everyone will be quietly and transparently migrated to the CardDAV backend. We can then start to clean up a lot of old code, always a nice thing to do.

If you’re trying the CardDAV beta test, we’d love to hear what you think. Let us know on twitter or by emailing carddavbeta@fastmail.com.

SSL certificates updated to SHA-256, RC4 disabled

Today we’re rolling out SHA-256 certificates. We announced this last month, and you can read that post for more information about why this is necessary.

At the same time, we’ve disabled the RC4 cipher suite. RC4 has long been considered broken and the browser security community recently started actively discouraging its use. The SSL Labs test penalises it, and Chrome has started presenting a low-priority warning.

All this means that we’re now get an A+ grade on the SSL Labs test, which is a good indicator that when it comes to our SSL/TLS configuration we’re pretty much in step with current industry best-practice.

If, like most of our users, you use the web client in a modern web browser, you won’t notice any difference. In older browsers and some IMAP/POP3/DAV/LDAP clients, you may start seeing disconnection problems if they don’t know how to handle SHA-256 certificates or rely on RC4. In these cases you’re encouraged to upgrade your clients and if necessary, contact your the author of your client for an update. In the meantime, you can use insecure.fastmail.com (web) and insecure.messagingengine.com (IMAP/POP/SMTP), both of support RC4 and have a SHA-1 certificate. As always, we highly discourage the use of these service names because they leave your data open to attack, and we may remove them in the future.

FastMail Advent 2014

Welcome to the inaugural FastMail Advent Blog. One post per day for the next 24 days.

The idea came from a reponse I made to a question on reddit about physical locations of servers and their impact on security.

I wrote that up in more detail, with links to blog posts about what FastMail does to address various categories of security risk, and suddenly found myself with something much too long for a single blog post. I wanted to split it up into separate posts, and then started thinking about frequency – and meanwhile my daughters were asking about advent calendars for the year, and it clicked.

We’ve been promising to blog more about some of our technology, and also about some of the less-well-known features – here was a perfect opportunity. You won’t just be hearing from me, I’m going to try to get everyone to write up something about their areas of expertise.

There’s a fine internet tradition in what we’re doing here, check out the perl advent calendar for example. They’ve been doing it for years.

I will be updating this blog post with links to every day as they are posted:

  1. Email Search System
  2. Security – Confidentiality, Integrity and Availability
  3. Push it real good
  4. Standalone Mail Servers
  5. Security – Integrity
  6. User authentication
  7. Automated installation
  8. Squire: FastMail’s rich text editor
  9. Email Authentication
  10. Security – Availability
  11. FastMail Support
  12. FastMail’s MySQL Replication: Multi-Master, Fault Tollerance, Performance. Pick Any Three
  13. FastMail DNS hosting
  14. On Duty!
  15. Putting the fast in FastMail: Loading your mailbox quickly
  16. Security – Confidentiality
  17. Testing
  18. Billing and Payments — a potted history
  19. Mailr
  20. Open-sourcing OvertureJS – the JS lib that powers FastMail
  21. File Storage
  22. CardDAV Beta Release
Posted in Advent 2014, News. Comments Off

FastMail app for iOS and Android now available

Today we’re proud to announce the release of the FastMail app for your iPhone, iPad, iPod and Android devices. You can get it right now from the App Store (iOS) or the Play Store (Android).

 Download on the App StoreGet it on Google Play

Our apps have been designed to combine our lightning-fast mobile web app with device features normally only available to native apps, most notably push notifications.

iOS notificationAndroid notification

On Android, you’ll even find support for your smartwatch!

Android Wear notificationPebble notification

More information about the FastMail app is available in our help.

Posted in News. Comments Off

FastMail has moved to fastmail.com, @fastmail.com email addresses now available

As discussed in a blog post earlier this week, we’ve now moved FastMail to fastmail.com. This means when you go to https://www.fastmail.fm, you’ll immediately be redirected to https://www.fastmail.com.

Does this affect my existing address or aliases?

Not at all, they will continue to function exactly as before. The only difference is the web address you’ll see in your browser when you log in to our website. This applies to all domains we host, not just @fastmail.fm.

How can I get an @fastmail.com email address?

With the exception of legacy guest and member accounts, you can add an alias (additional address) to your account, or you can rename your account to a new username @fastmail.com right now. Just go to https://www.fastmail.com, login to your account and go to Advanced -> Aliases to add an alias, or Advanced -> Rename account to rename your account.

All addresses are available on a first come, first served basis. We decided on this approach because we already offer many domains, so there might be joeblogs@fastmail.fm, joeblogs@fastmail.us, joeblogs@fastmail.net, joeblogs@myfastmail.com, joeblogs@eml.cc, etc. and we don’t think any particular user and any particular domain should get priority over another.

In the interests of fairness, we are only allowing each account to register one alias @fastmail.com. New users will be able to sign up an address @fastmail.com as well.

Email client users (e.g. Thunderbird, Apple Mail, Outlook, etc)

If you access your email through an email client, there’s no change. Everything will continue to work exactly as before.

Posted in News. Comments Off

FastMail is moving to fastmail.com

On Thursday, 23rd October 2014, we are moving the main FastMail website from fastmail.fm to fastmail.com. We intend to make the transition as seamless as possible, but we wanted to give you advance warning. Below are some more details for users regarding this change:

Email client users (e.g. Thunderbird, Apple Mail, Outlook, etc)

If you access your email through an email client, there’s no change. Everything will continue to work exactly as before.

Web interface users

If you use our web interface, from Thursday when you go to fastmail.fm you will be redirected automatically to fastmail.com. Any existing sessions will be transferred across, so if you were logged in at fastmail.fm, you’ll be logged in at fastmail.com. The only difference you should see is in the address bar in your browser.

Password manager users

If your password is normally filled in automatically for you by your browser or password manager, you’ll need to make sure you know what it is. For security reasons most password managers will only fill in your password on the domain where it was first used, and since we’re moving domains from fastmail.fm to fastmail.com, they’ll fail to work automatically. If you don’t know what your password is, we’ve got instructions on how to find it in all major browsers. Your password manager should prompt to save it again the first time you log in at fastmail.com, so don’t worry, you still won’t have to memorise it!

Does this affect my @fastmail.fm email address?

Not at all, this will continue to function exactly as before. The only difference is the web address you’ll see in your browser when you log in to our website.

How can I get an @fastmail.com email address?

With the exception of legacy guest and member accounts, you will be able to add an alias (additional address) to your account, or you will be able to rename your account to a new username @fastmail.com.

In the interests of fairness, we are only allowing each account to register one alias @fastmail.com. New users will be able to sign up an address @fastmail.com as well. All addresses will be available on a first come, first served basis, starting as soon as the transition to fastmail.com occurs.

When exactly will @fastmail.com email addresses become available?

An exact time on Thursday hasn’t been decided yet. Please keep an eye on this blog for further details.

Posted in News. Comments Off

New anti-phishing feature, all official FastMail emails have green tick mark

We’ve just rolled out a new feature that should help users identify official FastMail emails and avoid fake phishing emails.

All future official FastMail emails should now have a green tick next to them in the mailbox listing and when viewing the email/conversation. They look like this:

Screen Shot 2014-10-14 at 10.14.37 pm

Screen Shot 2014-10-14 at 10.17.52 pm

Users should be careful of any future emails that claim to be from FastMail that don’t have the green tick. These are almost certainly phishing emails that aim to steal your login details. Just report them as spam.

Note that the tick will only appear on future official FastMail emails, not existing ones. Also it only appears in the current web interface, not the classic web interface and not in external email clients (e.g. Outlook, Thunderbird, Mac Mail, etc)

Posted in News. Comments Off
Follow

Get every new post delivered to your Inbox.

Join 5,842 other followers