CalDav groups/categories not syncing

[ThibG]

Hi,
I am using DAVdroid for a while across several devices, using radicale as my CardDAV/CalDAV server, and recently noticed that my “new” phone did not synchronize groups.
Manually adding a group in the vCard files on the server and syncing does not create a group on my phone, and creating groups on my phone then syncing overwrites the categories for the particular contact with the newly-created group, discarding any group that that contact had.

DAVdroid version is 1.8.1-ose from FDroid,
Android version is 7.1.2 (LineageOS 14.1)

Steps to reproduce

1. Synchronize contacts that have groups set as the CATEGORIES of a remote vCard 3 contact.

Expected result

Contacts are created and groups corresponding to the CATEGORIES are created within the “Contacts” Android app with the appropriate members.

Actual result

Contacts are created, but no groups are created, and creating a new group overwrites the members’ CATEGORIES vCard field.

Logs

Trimmed down and anonymized logs regarding the addition of a (remotely-edited) contact into Group1 and Group2:

2017-10-17 16:41:12 225 [syncadapter.ContactsSyncManager] Updating [[CHANGED_CONTACT]].vcf in local address book
	PARAMETER #0 = at.bitfire.vcard4android.Contact@fe764d3[addresses=[],anniversary=<null>,birthDay=ezvcard.property.Birthday [ group=null | parameters={} | text=null | date=[[REDACTED]] | dateHasTime=false | partialDate=null ],categories=[Group1, Group2],displayName=[[REDACTED]],emails=[LabeledProperty(property=ezvcard.property.Email [ group=null | parameters={TYPE=[work]} | value=[[REDACTED]] ], label=null)],familyName=[[REDACTED]],givenName=[[REDACTED]],group=false,impps=[],jobDescription=<null>,jobTitle=<null>,members=[],middleName=<null>,nickName=<null>,note=<null>,organization=<null>,phoneNumbers=[LabeledProperty(property=ezvcard.property.Telephone [ group=null | parameters={TYPE=[cell]} | uri=null | text=[[REDACTED]] ], label=null)],phoneticFamilyName=<null>,phoneticGivenName=<null>,phoneticMiddleName=<null>,prefix=<null>,relations=[],suffix=<null>,uid=[[CHANGED_CONTACT]],unknownProperties=BEGIN:VCARD
VERSION:3.0
PRODID:ez-vcard 0.10.3
X-RADICALE-NAME:[[CHANGED_CONTACT]].vcf
END:VCARD
,urls=[]]
2017-10-17 16:41:12 225 [AndroidContact] Built RawContact data row
	PARAMETER #0 = mType: 2, mUri: content://com.android.contacts/raw_contacts/47?account_name=addressbook.vcf%20(thib%20Vg)&account_type=at.bitfire.davdroid.address_book&caller_is_syncadapter=true, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: sync2=d59e06c38ccf89c8b93cbc7adbbbdce0 deleted=0 sync1=[[CHANGED_CONTACT]] sourceid=[[CHANGED_CONTACT]].vcf dirty=0, mValuesBackReferences: null, mSelectionArgsBackReferences: null
2017-10-17 16:41:12 225 [AndroidContact] Built StructuredName data row
	PARAMETER #0 = mType: 1, mUri: content://com.android.contacts/data?account_name=addressbook.vcf%20(thib%20Vg)&account_type=at.bitfire.davdroid.address_book&caller_is_syncadapter=true, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: data9=null raw_contact_id=47 data5=null data8=null data1=[[CONTACT_NAME]] data6=null data2=[[CONTACT_FIRST_NAME]] data4=null data3=[[CONTACT_SURNAME]] data7=null mimetype=vnd.android.cursor.item/name, mValuesBackReferences: null, mSelectionArgsBackReferences: null
2017-10-17 16:41:12 225 [AndroidContact] Built Phone data row
	PARAMETER #0 = mType: 1, mUri: content://com.android.contacts/data?account_name=addressbook.vcf%20(thib%20Vg)&account_type=at.bitfire.davdroid.address_book&caller_is_syncadapter=true, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: raw_contact_id=47 is_primary=0 data1=[[REDACTED]] data2=2 data3=null is_super_primary=0 mimetype=vnd.android.cursor.item/phone_v2, mValuesBackReferences: null, mSelectionArgsBackReferences: null
2017-10-17 16:41:12 225 [AndroidContact] Built Email data row
	PARAMETER #0 = mType: 1, mUri: content://com.android.contacts/data?account_name=addressbook.vcf%20(thib%20Vg)&account_type=at.bitfire.davdroid.address_book&caller_is_syncadapter=true, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: raw_contact_id=47 is_primary=0 data1=[[REDACTED]] data2=2 data3=null is_super_primary=0 mimetype=vnd.android.cursor.item/email_v2, mValuesBackReferences: null, mSelectionArgsBackReferences: null
2017-10-17 16:41:12 225 [AndroidContact] Built Event data row
	PARAMETER #0 = mType: 1, mUri: content://com.android.contacts/data?account_name=addressbook.vcf%20(thib%20Vg)&account_type=at.bitfire.davdroid.address_book&caller_is_syncadapter=true, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: raw_contact_id=47 data1=1990-05-15 data2=3 mimetype=vnd.android.cursor.item/contact_event, mValuesBackReferences: null, mSelectionArgsBackReferences: null
2017-10-17 16:41:12 225 [BatchOperation] Committing 7 operations …
2017-10-17 16:41:12 225 [BatchOperation] Running operations 0 to 6
2017-10-17 16:41:13 225 [BatchOperation] … 12 record(s) affected
2017-10-17 16:41:13 225 [AndroidContact] Processing photo
2017-10-17 16:41:13 225 [AndroidContact] Inserting photo BLOB for raw contact 47
2017-10-17 16:41:13 225 [syncadapter.ContactsSyncManager] Removing contact group memberships
2017-10-17 16:41:13 225 [syncadapter.ContactsSyncManager] Adding membership in group Group1 (8)
2017-10-17 16:41:13 225 [syncadapter.ContactsSyncManager] Adding membership in group Group2 (13)
2017-10-17 16:41:13 225 [BatchOperation] Committing 5 operations …
2017-10-17 16:41:13 225 [BatchOperation] Running operations 0 to 4
2017-10-17 16:41:13 225 [BatchOperation] … 8 record(s) affected
2017-10-17 16:41:13 225 [resource.LocalContact] Calculated data hash = 362565551, group memberships hash = 21
2017-10-17 16:41:13 225 [resource.LocalContact] Storing contact hash = 362565562
2017-10-17 16:41:13 225 [syncadapter.SyncManager] Post-processing
2017-10-17 16:41:13 225 [syncadapter.ContactsSyncManager] Removing empty groups
2017-10-17 16:41:13 225 [resource.LocalAddressBook] Deleting group
	PARAMETER #0 = at.bitfire.davdroid.resource.LocalGroup@4706810[addressBook=at.bitfire.davdroid.resource.LocalAddressBook@ff15f09,contact=<null>,eTag=<null>,fileName=<null>,id=12]
2017-10-17 16:41:13 225 [syncadapter.SyncManager] Saving sync state
2017-10-17 16:41:13 225 [syncadapter.SyncManager] Saving CTag="37ee57656f90d531c2308c820a09891b"
2017-10-17 16:41:13 225 [syncadapter.ContactsSyncAdapterService] Contacts sync complete
2017-10-17 16:41:13 225 [syncadapter.SyncAdapterService] Sync for com.android.contacts complete
2017-10-17 16:41:23 1 [model.Settings] Received broadcast: re-initializing settings (logger)
2017-10-17 16:41:23 1 [log.Logger] Verbose logging: false```

[rfc2822]

Please provide steps to reproduce, DAVdroid debug info and verbose DAVdroid logs etc as described in [READ BEFORE POSTING] What’s required to diagnose a problem.

[ThibG]

Updated with trimmed down logs showing the process of adding a contact to two groups. This does not look like a CardDav protocol issue, but rather an issue with the Android side of things.

[rfc2822]

Can you please provide the debug info and the server capabilities, too? Should look like that in the logs:

10-19 13:09:59.431 5644-6588/at.bitfire.davdroid:sync I/davdroid: [syncadapter.ContactsSyncManager] Server advertises VCard/4 support: true
10-19 13:09:59.447 5644-6588/at.bitfire.davdroid:sync I/davdroid: [syncadapter.ContactsSyncManager] Contact group method: CATEGORIES

I have tried Android 7 and synchronizing CATEGORIES groups, and I can’t reproduce your problem. Also, I can’t see any suspicious entries in your logs …

[ThibG]

Sorry for the late reply. Here are the logs regarding the server capabilities from the same log:

2017-10-17 16:41:09 225 [syncadapter.ContactsSyncManager] Server advertises VCard/4 support: false
2017-10-17 16:41:09 225 [syncadapter.ContactsSyncManager] Contact group method: CATEGORIES

I’m not the only one to experience this issue: in fact, I reported it after multiple people complained about it online.
I am sorry I can’t see how to reproduce this issue

To make it clear, I think we all use Android 7.1.2 (LineageOS 14.1) with the stock “Contacts” app, and the issue is about synchronizing groups/contacts created remotely, not synchronizing groups created from the phone itself.