DAVdroid uses the calendar-color property returned by the server (with green as fallback).
Set this property on your server and DAVdroid will import the calendars with the colors.
As @untitaker said, the colors are only imported when setting up the account. If you’re interested in synchronizing color changes from the server, please follow up in #219. If you just want to set your calendar colors, you may use the Calendarcolor app.