FN not syncing with display_name

[equaeghe]

The FN field of my vcards is not being synced with the Android display_name. Instead, the display name is the full structured N, where ‘;’ is replaced by a space. I use DAVx⁵ with FastMail. I use vdirsyncer and checked that the FN is indeed (usually) in ‘First Last’ format and not in ‘Dr. First Middle Last’ format. I expected FN to sync with display_name based on the documentation.

I assume that the ‘Dr. First Middle Last’ format is generated by the Contact App or some other part of Android. Other fields do sync correctly.

I have downloaded the contacts.db file and looked at its view_data view. There in its DATA1 column (cf. Android docs; display_name column in the actual raw_contact table), I find ‘Dr. First Middle Last’ being used.

I wanted to check in the logs of a sync in which I change something in an affected contact. So I did and the modified contact was synced (change visible in Contacts app). However, even though logging is active and I have the notification with the share button, no log file seems to be generated and shareable (I tried to share by mailing, owncloud, saving to storage, but all I managed to do is generate errors or crashes in those apps.)

I’m using version 2.6 of the app, but actually this has never worked AFAIR. I’m using Android 8.1.0/LineageOS 15.1.

[devvv4ever]

Which contacts app do you use?
Did you have a look at our FAQ here: https://www.davx5.com/faq/entering-structured-addresses ?

[equaeghe]

@devvv4ever said in FN not syncing with display_name:

Which contacts app do you use?

I use the LineageOS Contacts app, version 1.7.21, which I think is AOSP Contacts. It does not allow editing the display name itself, but it does allow structured entry of the full name and any titles (I do not usually use the app for entry, though). Unless the Contacts app overwrites the display_name field upon every sync, which I consider unlikely, I do not see how the database values could be as they are unless they are not properly synced into the database.

Did you have a look at our FAQ here: https://www.davx5.com/faq/entering-structured-addresses ?

My report is about FN, not about addresses. Also, I do not enter contacts on my phone. I do that in the Fastmail web interface or in the vcards vdirsyncer syncs.

[rfc2822]

I do not understand the problem. Can you please provide steps so that I can reproduce the problem? We have a FastMail test account.

By the way, you can see the Android / vCard conversion logic here:

• vCard → Android: https://gitlab.com/bitfireAT/vcard4android/blob/c7dcf57ec4e9fc296cc2ee9b800b29a6fba4c0a7/src/main/java/at/bitfire/vcard4android/AndroidContact.kt#L190
• Android → vCard: https://gitlab.com/bitfireAT/vcard4android/blob/c7dcf57ec4e9fc296cc2ee9b800b29a6fba4c0a7/src/main/java/at/bitfire/vcard4android/AndroidContact.kt#L656

However, even though logging is active and I have the notification with the share button, no log file seems to be generated and shareable (I tried to share by mailing, owncloud, saving to storage, but all I managed to do is generate errors or crashes in those apps.)

This is a problem that we hear from time to time, too, but I can’t imagine its cause and I was never able to reproduce it. Would it be possible that you use adb to get more information about both 1. the display name and 2. the empty logs?

[equaeghe]

@rfc2822 said in FN not syncing with display_name:

I do not understand the problem. Can you please provide steps so that I can reproduce the problem? We have a FastMail test account.

1. Create contact android-side:

   prefix = Dr.
   given = First
   middle = Middle
   family = last
   postfix. = Jr.

   nick = FMS

2. Sync with DAVx⁵ to server

3. Sync some place where you can edit it (Fastmail interface does not allow editing FN; I use vdirsyncer) and view synced contact:

   BEGIN:VCARD
   VERSION:3.0
   PRODID:+//IDN bitfire.at//DAVx5/2.6-gplay ez-vcard/0.10.5
   UID:442f173d-651c-46f6-b28e-5f01c80c69b2
   FN:Dr. First Middle Last\, Jr.
   N:Last;First;Middle;Dr.;Jr.
   NICKNAME:FMS
   REV:2019-10-30T14:11:30Z
   END:VCARD
   

4. Edit contact to change FN and NICKNAME (as a canary visible in Android Contacts app):

   BEGIN:VCARD
   VERSION:3.0
   FN:First Last
   N:Last;First;Middle;Dr.;Jr.
   NICKNAME:Junior
   PRODID:Manual Edit/Erik Quaeghebeur
   REV:2019-10-30T14:13:00Z
   UID:442f173d-651c-46f6-b28e-5f01c80c69b2
   END:VCARD
   

5. Sync back to server (using vdirsyncer or so)

6. Check by downloading vcard from Fastmail interface that change has propagated (success for me; the vcard in step 4 is actually the one provided by Fastmail’s interface after my edit).

7. Sync with DAVx⁵ to Android device

8. Check by viewing contact in Contacts app (nick changed, but display name hasn’t)

9. Export vcard from Contacts app interface for some extra info (vcard 2.1, the horror) :

   BEGIN:VCARD
   VERSION:2.1
   N:Last;First;Middle;Dr.;Jr.
   FN:First Last
   X-ANDROID-CUSTOM:vnd.android.cursor.item/nickname;Junior;1;;;;;;;;;;;;;
   END:VCARD
   

However, even though logging is active and I have the notification with the share button, no log file seems to be generated and shareable […]

This is a problem that we hear from time to time, too, but I can’t imagine its cause and I was never able to reproduce it.

I managed to get logs after restarting DAVx⁵. You can find them at https://paste.fedoraproject.org/paste/utMImkv0IJ2-JKxsdx6fLQ. TLDR: It’s not Fastmail’s fault and DAVx⁵ correctly assigns FN to data1. So I fear it may be Android enforcing display_name as an unstructured representation of the rest of the name fields. I do hope it is not the case and DAVx⁵ can actually still make sure FN is really propagated to display_name (but what about display_name_alt then…).

I hope this all clarifies the issue I encounter for you and allows you assess the issue I have.

[rfc2822]

I see. Thanks for the logs. Here you can see:

FN:First Last

so the display name should be “First Last”. That is what DAVx⁵ inserts to the Android database:

2019-10-30 15:14:25 1077 [AndroidContact] Built StructuredName data row
PARAMETER #0 = mType: 1, mUri: content://com.android.contacts/data?caller_is_syncadapter=true&account_name=Default%20(REDACTED_PERCENT_ENCODED_ACCOUNT_EMAIL%20Tg)&account_type=at.bitfire.davdroid.address_book, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: raw_contact_id=678 data9=null data8=null data7=null data6=Jr. data5=Middle data4=Dr. data3=Last data2=First data1=First Last mimetype=vnd.android.cursor.item/name, mValuesBackReferences: null, mSelectionArgsBackReferences: null

Note that data1 is “First Last”, and data1 is the display_name for this row type. So it seems that DAVx⁵ inserts the correct value. You can also verify it with adb shell content query --uri content://com.android.contacts/raw_contacts as described on https://www.davx5.com/faq/how-to-debug.

Are there more raw contacts for the same contact, i.e. do you have other contact sync adapters (Google, WhatsApp etc.)? In this case, the other display_name may be a result of raw contact merging.

[equaeghe]

@rfc2822 said in FN not syncing with display_name:

Are there more raw contacts for the same contact, i.e. do you have other contact sync adapters (Google, WhatsApp etc.)? In this case, the other display_name may be a result of raw contact merging.

Well, I looked at the sqlite database again, and there are tables/views with data1 columns and with display_name columns. These are different. My guess is that the Contacts app uses the display_name column value (I am actually quite sure of this). Furthermore, other apps (such as FairEmail) also use this column for autocompleting email addresses.

Then there is Signal, it uses the existing contact entries to find phone numbers whose owners have Signal installed. It then creates its own entry, with a completely messed up set of name columns, including the data1 one, which looks like display_name now.

So I guess other apps are accessing the contacts’ display name through ContactsContract.ContactsColumns.DISPLAY_NAME, which is not guaranteed to be the same as data1, as it doesn’t access the data table, but the raw_contacts table (via a view aka join). So the existence of two, possibly different fields for display names is creating the issue I have. If DAVx⁵ can force raw_contacts.display_name to be equal to data.data1, then maybe it should. But my fear is that Android is single-handedly forming the raw_contacts.display_name and raw_contacts.display_name_alt in a pre-set way based on the other name fields. It looks like a mess…

[equaeghe]

@equaeghe said in FN not syncing with display_name:

If DAVx⁵ can force raw_contacts.display_name to be equal to data.data1, then maybe it should.

I think there is: DISPLAY_NAME_PRIMARY leaving DisplayNameSources UNDEFINED. (These are links to the API.)

[rfc2822]

@equaeghe said in FN not syncing with display_name:

Well, I looked at the sqlite database again, and there are tables/views with data1 columns and with display_name columns. These are different. My guess is that the Contacts app uses the display_name column value (I am actually quite sure of this). Furthermore, other apps (such as FairEmail) also use this column for autocompleting email addresses.

The display_name is a column of the (automatically aggregated) contacts table. If an app shows /contacts/, it will usually use the contacts table and thus the display_names of the contacts.

There is important information and especially two schematics in the API docs:

• https://developer.android.com/guide/topics/providers/contacts-provider#InformationTypes
• https://developer.android.com/guide/topics/providers/contacts-provider#Sources

Then there is Signal, it uses the existing contact entries to find phone numbers whose owners have Signal installed. It then creates its own entry, with a completely messed up set of name columns, including the data1 one, which looks like display_name now.

It’s one of the installed sync adapters (which manage raw_contacts), exactly on the same level as DAVx⁵.

So I guess other apps are accessing the contacts’ display name through ContactsContract.ContactsColumns.DISPLAY_NAME, which is not guaranteed to be the same as data1,

Exactly. Android will merge all the raw contacts (DAVx⁵ and Signal raw_contacts with data1) to one contact (contact table with display_name).

So the existence of two, possibly different fields for display names is creating the issue I have.

This is what I have meant by /raw contact merging/. Thanks for the detailled verification, it is really helpful!

If DAVx⁵ can force raw_contacts.display_name to be equal to data.data1, then maybe it should. But my fear is that Android is single-handedly forming the raw_contacts.display_name and raw_contacts.display_name_alt in a pre-set way based on the other name fields. It looks like a mess…

I’m not sure about that. DISPLAY_NAME_PRIMARY is a column of the (automatically managed) contacts table, which should – as I understand it – not be touched by sync adapters. Android will then merge the available raw contacts and choose an appropriate display name. It seems that in your case, Android chooses Signal’s display name.

Now it would by interesting how Android merges the raw contacts and why it prefers Signal’s display name in your case.

I’d rather not priorize DAVx⁵ values in the contacts table, even if it would be possible, because it might not always be the desired result. For instance, if you have DAVx⁵ and a Google account with two raw_contacts of the same contact, the display name on Google might be the more important.

[equaeghe]

@rfc2822

I do not entirely agree. Raw contact merging is not the cause of the issue that I encounter. (It is a factor leading to confusion for sure.)

The following is based on inspection of actual data of the actual contacts database extracted from my phone.

There are three tables of relevance here, data, raw_contacts, and contacts. There is also a view (aka join) of interest, view_contacts. (I have anonymized the values below by replacing the true first name by ‘First’ etc.)

Extract from data (all rows where data1 contains ‘First’):

raw_contact_id | data1                          | data2             | data3 | data4 | data5
           587 | dr. First Middle1,Middle2 Last | dr. First Middle1 | Last  |       | Middle2
           578 | First Last                     | First             | Last  | dr.   | Middle1 Middle2

‘578’ is the (correct) DAVx⁵ one and ‘587’ the messed up Signal one. Notice that the right value (from FN) is in data1.

Extract from raw_contacts (all rows where display_name contains ‘First’):

_id | contact_id | display_name                   | display_name_alt                | display_name_source
578 |        578 | dr. First Middle1 Middle2 Last | dr. Last, First Middle1 Middle2 | 40
587 |        578 | dr. First Middle1 Middle2 Last | Last, dr. First Middle1 Middle2 | 40

Notice that for ‘578’ a value different from data1 is in display_name. This is the value that is being shown by the Contacts app and used by my Email app (emails are only defined for ‘578’). I would like this value to coincide with FN. I think they get this value from view_contacts (all rows where display_name contains ‘First’):

_id | display_name_source | display_name                   | display_name_alt
578 | 40                  | dr. First Middle1 Middle2 Last | dr. Last, First Middle1 Middle2

I think it is accessed using DISPLAY_NAME_PRIMARY (cf. docs).

For completeness, from the contacts table:

_id | name_raw_contacts_id 
578 | 578

which, I think means that at least for this contact, DAVx⁵ provides the name shown. (However that was decided would be interesting, but not relevant for this issue.)

So, yes, I think that if display_name in the raw_contacts table is set equal to data1/FN (for DAVx⁵’s row, ‘578’ in my example), my issue would be solved. However, I do not know how or even if this can be set by the sync adapter; the docs for DISPLAY_NAME_PRIMARY are not encouraging:

The standard text shown as the contact’s display name, based on the best available information for the contact (for example, it might be the email address if the name is not available). The information actually used to compute the name is stored in DISPLAY_NAME_SOURCE.

A contacts provider is free to choose whatever representation makes most sense for its target market. For example in the default Android Open Source Project implementation, if the display name is based on the structured name and the structured name follows the Western full-name style, then this field contains the “given name first” version of the full name.

So it seems to be the provider implementation (AOSP/LineageOS for me) that decides this, and not the sync adapter. (I think it is crazy that a user apparently has no control over the display name.) But perhaps I missed some method somewhere that can set the display name. If such a method were to exist, it would be logical that DAVx⁵ uses it to let data1/FN be used. If no such method exists, data1 is completely superfluous (for the AOSP/LineageOS implementation). I wonder whether Google and Samsung modify the contacts provider from the AOSP one for their devices?

Enough for tonight…

[rfc2822]

I’ll have a look at display_name in raw_contacts tomorrow. By the way, the source code of the AOSP Contacts Provider is available on android.googlesource.com (Github mirror). I guess the answers the can be found there…

[equaeghe]

@rfc2822 said in FN not syncing with display_name:

[…] By the way, the source code of the AOSP Contacts Provider is available on android.googlesource.com (Github mirror). I guess the answers the can be found there…

The function responsible for constructing display_name ignores DATA1 when the MIME type is a structured name. All hope is lost… (User workarounds may still be possible, by having a second contacts source DAVx⁵ syncs which just contains ‘First’ and ‘Last’ in N, getting it merged with the full one and getting its display name to be the one used…)

[rfc2822]

Ok… If you find something that can be done by DAVx⁵, please just post here.

[rfc2822]

By the way, did you see in adb why sharing the logs didn’t work? Either when turning the log feature on or when actually sharing.

[equaeghe]

@rfc2822 said in FN not syncing with display_name:

By the way, did you see in adb why sharing the logs didn’t work? Either when turning the log feature on or when actually sharing.

I never used adb. I ‘solved’ it my restarting DAVx⁵ before logging, as then it always worked.