Thanks anyway 🙂 And have a good day !
FN not syncing with display_name
-
The FN field of my vcards is not being synced with the Android display_name. Instead, the display name is the full structured N, where ‘;’ is replaced by a space. I use DAVx⁵ with FastMail. I use vdirsyncer and checked that the FN is indeed (usually) in ‘First Last’ format and not in ‘Dr. First Middle Last’ format. I expected FN to sync with display_name based on the documentation.
I assume that the ‘Dr. First Middle Last’ format is generated by the Contact App or some other part of Android. Other fields do sync correctly.
I have downloaded the
contacts.db
file and looked at itsview_data
view. There in itsDATA1
column (cf. Android docs;display_name
column in the actualraw_contact
table), I find ‘Dr. First Middle Last’ being used.I wanted to check in the logs of a sync in which I change something in an affected contact. So I did and the modified contact was synced (change visible in Contacts app). However, even though logging is active and I have the notification with the share button, no log file seems to be generated and shareable (I tried to share by mailing, owncloud, saving to storage, but all I managed to do is generate errors or crashes in those apps.)
I’m using version 2.6 of the app, but actually this has never worked AFAIR. I’m using Android 8.1.0/LineageOS 15.1.
-
Which contacts app do you use?
Did you have a look at our FAQ here: https://www.davx5.com/faq/entering-structured-addresses ? -
@devvv4ever said in FN not syncing with display_name:
Which contacts app do you use?
I use the LineageOS Contacts app, version 1.7.21, which I think is AOSP Contacts. It does not allow editing the display name itself, but it does allow structured entry of the full name and any titles (I do not usually use the app for entry, though). Unless the Contacts app overwrites the
display_name
field upon every sync, which I consider unlikely, I do not see how the database values could be as they are unless they are not properly synced into the database.Did you have a look at our FAQ here: https://www.davx5.com/faq/entering-structured-addresses ?
My report is about FN, not about addresses. Also, I do not enter contacts on my phone. I do that in the Fastmail web interface or in the vcards vdirsyncer syncs.
-
I do not understand the problem. Can you please provide steps so that I can reproduce the problem? We have a FastMail test account.
By the way, you can see the Android / vCard conversion logic here:
- vCard → Android: https://gitlab.com/bitfireAT/vcard4android/blob/c7dcf57ec4e9fc296cc2ee9b800b29a6fba4c0a7/src/main/java/at/bitfire/vcard4android/AndroidContact.kt#L190
- Android → vCard: https://gitlab.com/bitfireAT/vcard4android/blob/c7dcf57ec4e9fc296cc2ee9b800b29a6fba4c0a7/src/main/java/at/bitfire/vcard4android/AndroidContact.kt#L656
However, even though logging is active and I have the notification with the share button, no log file seems to be generated and shareable (I tried to share by mailing, owncloud, saving to storage, but all I managed to do is generate errors or crashes in those apps.)
This is a problem that we hear from time to time, too, but I can’t imagine its cause and I was never able to reproduce it. Would it be possible that you use adb to get more information about both 1. the display name and 2. the empty logs?
-
@rfc2822 said in FN not syncing with display_name:
I do not understand the problem. Can you please provide steps so that I can reproduce the problem? We have a FastMail test account.
-
Create contact android-side:
prefix = Dr.
given = First
middle = Middle
family = last
postfix. = Jr.nick = FMS
-
Sync with DAVx⁵ to server
-
Sync some place where you can edit it (Fastmail interface does not allow editing FN; I use vdirsyncer) and view synced contact:
BEGIN:VCARD VERSION:3.0 PRODID:+//IDN bitfire.at//DAVx5/2.6-gplay ez-vcard/0.10.5 UID:442f173d-651c-46f6-b28e-5f01c80c69b2 FN:Dr. First Middle Last\, Jr. N:Last;First;Middle;Dr.;Jr. NICKNAME:FMS REV:2019-10-30T14:11:30Z END:VCARD
-
Edit contact to change FN and NICKNAME (as a canary visible in Android Contacts app):
BEGIN:VCARD VERSION:3.0 FN:First Last N:Last;First;Middle;Dr.;Jr. NICKNAME:Junior PRODID:Manual Edit/Erik Quaeghebeur REV:2019-10-30T14:13:00Z UID:442f173d-651c-46f6-b28e-5f01c80c69b2 END:VCARD
-
Sync back to server (using vdirsyncer or so)
-
Check by downloading vcard from Fastmail interface that change has propagated (success for me; the vcard in step 4 is actually the one provided by Fastmail’s interface after my edit).
-
Sync with DAVx⁵ to Android device
-
Check by viewing contact in Contacts app (nick changed, but display name hasn’t)
-
Export vcard from Contacts app interface for some extra info (vcard 2.1, the horror) :
BEGIN:VCARD VERSION:2.1 N:Last;First;Middle;Dr.;Jr. FN:First Last X-ANDROID-CUSTOM:vnd.android.cursor.item/nickname;Junior;1;;;;;;;;;;;;; END:VCARD
However, even though logging is active and I have the notification with the share button, no log file seems to be generated and shareable […]
This is a problem that we hear from time to time, too, but I can’t imagine its cause and I was never able to reproduce it.
I managed to get logs after restarting DAVx⁵. You can find them at https://paste.fedoraproject.org/paste/utMImkv0IJ2-JKxsdx6fLQ. TLDR: It’s not Fastmail’s fault and DAVx⁵ correctly assigns
FN
todata1
. So I fear it may be Android enforcingdisplay_name
as an unstructured representation of the rest of the name fields. I do hope it is not the case and DAVx⁵ can actually still make sureFN
is really propagated todisplay_name
(but what aboutdisplay_name_alt
then…).I hope this all clarifies the issue I encounter for you and allows you assess the issue I have.
-
-
I see. Thanks for the logs. Here you can see:
FN:First Last
so the display name should be “First Last”. That is what DAVx⁵ inserts to the Android database:
2019-10-30 15:14:25 1077 [AndroidContact] Built StructuredName data row
PARAMETER #0 = mType: 1, mUri: content://com.android.contacts/data?caller_is_syncadapter=true&account_name=Default%20(REDACTED_PERCENT_ENCODED_ACCOUNT_EMAIL%20Tg)&account_type=at.bitfire.davdroid.address_book, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: raw_contact_id=678 data9=null data8=null data7=null data6=Jr. data5=Middle data4=Dr. data3=Last data2=First data1=First Last mimetype=vnd.android.cursor.item/name, mValuesBackReferences: null, mSelectionArgsBackReferences: nullNote that
data1
is “First Last”, anddata1
is the display_name for this row type. So it seems that DAVx⁵ inserts the correct value. You can also verify it withadb shell content query --uri content://com.android.contacts/raw_contacts
as described on https://www.davx5.com/faq/how-to-debug.Are there more raw contacts for the same contact, i.e. do you have other contact sync adapters (Google, WhatsApp etc.)? In this case, the other
display_name
may be a result of raw contact merging. -
@rfc2822 said in FN not syncing with display_name:
Are there more raw contacts for the same contact, i.e. do you have other contact sync adapters (Google, WhatsApp etc.)? In this case, the other
display_name
may be a result of raw contact merging.Well, I looked at the sqlite database again, and there are tables/views with
data1
columns and withdisplay_name
columns. These are different. My guess is that the Contacts app uses thedisplay_name
column value (I am actually quite sure of this). Furthermore, other apps (such as FairEmail) also use this column for autocompleting email addresses.Then there is Signal, it uses the existing contact entries to find phone numbers whose owners have Signal installed. It then creates its own entry, with a completely messed up set of name columns, including the
data1
one, which looks likedisplay_name
now.So I guess other apps are accessing the contacts’ display name through
ContactsContract.ContactsColumns.DISPLAY_NAME
, which is not guaranteed to be the same asdata1
, as it doesn’t access thedata
table, but theraw_contacts
table (via a view aka join). So the existence of two, possibly different fields for display names is creating the issue I have. If DAVx⁵ can forceraw_contacts.display_name
to be equal todata.data1
, then maybe it should. But my fear is that Android is single-handedly forming theraw_contacts.display_name
andraw_contacts.display_name_alt
in a pre-set way based on the other name fields. It looks like a mess… -
@equaeghe said in FN not syncing with display_name:
If DAVx⁵ can force
raw_contacts.display_name
to be equal todata.data1
, then maybe it should.I think there is:
DISPLAY_NAME_PRIMARY
leavingDisplayNameSources
UNDEFINED
. (These are links to the API.) -
@equaeghe said in FN not syncing with display_name:
Well, I looked at the sqlite database again, and there are tables/views with
data1
columns and withdisplay_name
columns. These are different. My guess is that the Contacts app uses thedisplay_name
column value (I am actually quite sure of this). Furthermore, other apps (such as FairEmail) also use this column for autocompleting email addresses.The
display_name
is a column of the (automatically aggregated)contacts
table. If an app shows /contacts/, it will usually use thecontacts
table and thus thedisplay_name
s of the contacts.There is important information and especially two schematics in the API docs:
- https://developer.android.com/guide/topics/providers/contacts-provider#InformationTypes
- https://developer.android.com/guide/topics/providers/contacts-provider#Sources
Then there is Signal, it uses the existing contact entries to find phone numbers whose owners have Signal installed. It then creates its own entry, with a completely messed up set of name columns, including the
data1
one, which looks likedisplay_name
now.It’s one of the installed sync adapters (which manage
raw_contacts
), exactly on the same level as DAVx⁵.So I guess other apps are accessing the contacts’ display name through
ContactsContract.ContactsColumns.DISPLAY_NAME
, which is not guaranteed to be the same asdata1
,Exactly. Android will merge all the raw contacts (DAVx⁵ and Signal
raw_contacts
withdata1
) to one contact (contact
table withdisplay_name
).So the existence of two, possibly different fields for display names is creating the issue I have.
This is what I have meant by /raw contact merging/. Thanks for the detailled verification, it is really helpful!
If DAVx⁵ can force
raw_contacts.display_name
to be equal todata.data1
, then maybe it should. But my fear is that Android is single-handedly forming theraw_contacts.display_name
andraw_contacts.display_name_alt
in a pre-set way based on the other name fields. It looks like a mess…I’m not sure about that.
DISPLAY_NAME_PRIMARY
is a column of the (automatically managed)contacts
table, which should – as I understand it – not be touched by sync adapters. Android will then merge the available raw contacts and choose an appropriate display name. It seems that in your case, Android chooses Signal’s display name.Now it would by interesting how Android merges the raw contacts and why it prefers Signal’s display name in your case.
I’d rather not priorize DAVx⁵ values in the
contacts
table, even if it would be possible, because it might not always be the desired result. For instance, if you have DAVx⁵ and a Google account with tworaw_contact
s of the samecontact
, the display name on Google might be the more important. -
I do not entirely agree. Raw contact merging is not the cause of the issue that I encounter. (It is a factor leading to confusion for sure.)
The following is based on inspection of actual data of the actual contacts database extracted from my phone.
There are three tables of relevance here,
data
,raw_contacts
, andcontacts
. There is also a view (aka join) of interest,view_contacts
. (I have anonymized the values below by replacing the true first name by ‘First’ etc.)Extract from
data
(all rows wheredata1
contains ‘First’):raw_contact_id | data1 | data2 | data3 | data4 | data5 587 | dr. First Middle1,Middle2 Last | dr. First Middle1 | Last | | Middle2 578 | First Last | First | Last | dr. | Middle1 Middle2
‘578’ is the (correct) DAVx⁵ one and ‘587’ the messed up Signal one. Notice that the right value (from
FN
) is indata1
.Extract from
raw_contacts
(all rows wheredisplay_name
contains ‘First’):_id | contact_id | display_name | display_name_alt | display_name_source 578 | 578 | dr. First Middle1 Middle2 Last | dr. Last, First Middle1 Middle2 | 40 587 | 578 | dr. First Middle1 Middle2 Last | Last, dr. First Middle1 Middle2 | 40
Notice that for ‘578’ a value different from
data1
is indisplay_name
. This is the value that is being shown by the Contacts app and used by my Email app (emails are only defined for ‘578’). I would like this value to coincide withFN
. I think they get this value fromview_contacts
(all rows wheredisplay_name
contains ‘First’):_id | display_name_source | display_name | display_name_alt 578 | 40 | dr. First Middle1 Middle2 Last | dr. Last, First Middle1 Middle2
I think it is accessed using
DISPLAY_NAME_PRIMARY
(cf. docs).For completeness, from the
contacts
table:_id | name_raw_contacts_id 578 | 578
which, I think means that at least for this contact, DAVx⁵ provides the name shown. (However that was decided would be interesting, but not relevant for this issue.)
So, yes, I think that if
display_name
in theraw_contacts
table is set equal todata1
/FN
(for DAVx⁵’s row, ‘578’ in my example), my issue would be solved. However, I do not know how or even if this can be set by the sync adapter; the docs forDISPLAY_NAME_PRIMARY
are not encouraging:The standard text shown as the contact’s display name, based on the best available information for the contact (for example, it might be the email address if the name is not available). The information actually used to compute the name is stored in
DISPLAY_NAME_SOURCE
.A contacts provider is free to choose whatever representation makes most sense for its target market. For example in the default Android Open Source Project implementation, if the display name is based on the structured name and the structured name follows the Western full-name style, then this field contains the “given name first” version of the full name.
So it seems to be the provider implementation (AOSP/LineageOS for me) that decides this, and not the sync adapter. (I think it is crazy that a user apparently has no control over the display name.) But perhaps I missed some method somewhere that can set the display name. If such a method were to exist, it would be logical that DAVx⁵ uses it to let
data1
/FN
be used. If no such method exists,data1
is completely superfluous (for the AOSP/LineageOS implementation). I wonder whether Google and Samsung modify the contacts provider from the AOSP one for their devices?Enough for tonight…
-
I’ll have a look at
display_name
inraw_contact
s tomorrow. By the way, the source code of the AOSP Contacts Provider is available on android.googlesource.com (Github mirror). I guess the answers the can be found there… -
@rfc2822 said in FN not syncing with display_name:
[…] By the way, the source code of the AOSP Contacts Provider is available on android.googlesource.com (Github mirror). I guess the answers the can be found there…
The function responsible for constructing
display_name
ignoresDATA1
when the MIME type is a structured name. All hope is lost… (User workarounds may still be possible, by having a second contacts source DAVx⁵ syncs which just contains ‘First’ and ‘Last’ inN
, getting it merged with the full one and getting its display name to be the one used…) -
Ok… If you find something that can be done by DAVx⁵, please just post here.
-
By the way, did you see in adb why sharing the logs didn’t work? Either when turning the log feature on or when actually sharing.
-
@rfc2822 said in FN not syncing with display_name:
By the way, did you see in adb why sharing the logs didn’t work? Either when turning the log feature on or when actually sharing.
I never used adb. I ‘solved’ it my restarting DAVx⁵ before logging, as then it always worked.