Maybe there is a contact that is corrupt or so. Without logs we can’t say whats wrong. You can activate the logging in the DAVdroid app settings.
Remove the account first, then add it again but don’t select an address book and don’t start the sync process yet. Start the logging after it and then select the address book and press the sync button in the account menu on the top. You can send the logs to firstname.lastname@example.org - then we can have a look at it.