http://blog.fastmail.com/2014/12/22/carddav-beta-release/
Today FastMail added (beta) support for CardDAV, and it seems to work fine with DAVdroid, except for one thing; groups.
I’m using a custom Android 4.4.4 ROM (AOSPA) with the patch mentioned in issue #48:
https://github.com/AOSPA/android_packages_apps_ContactsCommon/commit/574b9c7d6a37b32b1c038fa4a1e23fa6a37b518f
It might be something that my contacts/dialer app doesn’t support properly either, so if you can point me in the direction of one that is confirmed to support groups properly I’d test that as well.