Contacts not visible in Sony Contacts app



  • Since some time ago (unfortunately I'm not able to track this exactly) I'm unable to add new contacts.

    This happens both when adding contact in phone or in the CardDAV. When adding in phone, the contact is synced to CardDAV, but not shown in the phone. When it's added on CardDAV, DAVdroid downloads it.

    Not sure how to debug this or what logs might be useful. For example this is when adding contact in the phone:

    I/davdroid.DavSyncAdapter(29195): Performing sync for authority com.android.contacts
    D/davdroid.DavSyncAdapter(29195): Creating new DavHttpClient
    D/davdroid.DavSyncAdapter(29195): Server supports VCard version 3.0
    V/davdroid.URIUtils(29195): Normalized URL https://xxx/nijel/contacts/ -> https://xxx/nijel/contacts/
    D/davdroid.WebDavResource(29195): Using preemptive authentication (not compatible with Digest auth)
    I/davdroid.SyncManager(29195): Remotely removing 0 deleted resource(s) (if not changed)
    I/davdroid.SyncManager(29195): Uploading 1 new resource(s) (if not existing)
    E/SQLiteLog(31371): (284) automatic index on sqlite_sq_A2426EC0(STAT_DATA_ID)
    E/SQLiteLog(31371): (284) automatic index on sqlite_sq_AEE602E0(STAT_DATA_ID)
    V/davdroid.URIUtils(29195): Normalized URL 169118da-7ab0-48c2-b116-bcc2fa7c5de0.vcf -> 169118da-7ab0-48c2-b116-bcc2fa7c5de0.vcf
    D/davdroid.Collection(29195): Setting ETag of local resource 169118da-7ab0-48c2-b116-bcc2fa7c5de0.vcf to "ff823c6a93d03cd1c4e6ea3a644bfc80"
    D/davdroid.Collection(29195): Committing 1 operations
    

    It went just fine to the server:

    2015-06-17 13:01:51,928 - INFO: PUT request at /nijel/contacts/169118da-7ab0-48c2-b116-bcc2fa7c5de0.vcf received
    

    The server is Radicale 0.10, phone Xperia Z1 Compact with Android 5.0.2, but same happens with Xperia E4g with Android 4.4.4, both with DAVdroid 0.8.0.


  • developer

    Did you enable the DAVdroid address book to show in your Contacts app?



  • Yes as it shows DAVdroid contacts which were existing so far. I'm just unable to add new ones.



  • Okay, it seems to be caused by Sony contacts application (or rather by some update to it as it used to work fine). After installing True contacts I can see all of them. Also when configuring contacts to display in this application to include contacts without group, Sony contacts magically start to show them as well.

    So the problem here was that new contacts are added without group and the default contacts application not showing these. Not sure if something can be fixed on DAVdroid side for this...


  • developer

    So, could you finally configure the default Sony Contacts app to show contacts without group?



  • There is no configuration option for this in the Sony app, but it honors settings done in the other app. Also in past it was possible to add contacts without group to some, now it stopped to work. But still this is more annoyance of Sony than DAVdroid's.

    The only thing which could improve the situation on DAVdroid side is adding all synced contacts to a specific group, this way it would avoid unpleasant experience of contacts not being shown. But this is clearly a workaround for app limitation.



  • Hi,

    I am experiencing the same problem, with a Sony device (Z3 Compact) but Groups don't have anything to do with it, at least in my experience:

    When I add a new contact, a number of fields is shown: Name, Phonetic last name, Phonetic middle name, and Phonetic first name, but you can show other fields with a click. If you chose "Name", which is the logical one, what you type is ignored and not passed to SabreDav (the server I use). The contact is indeed created, but only the phone number is registered (both inside the Android device and the SabreDav server)... and if you browse inside the server and download the VCARD file, only the number field shows up.

    Instead, if you show other fields and choose "First Name", then that is properly registered both inside the phone and in the server and, curiously enough, the "Name" field is then populated with what you entered as "First Name". Further changes on "Name" are discarded if you keep "First Name" untouched.

    So, I think the problem is that DavDroid is not handling the "Name" field correctly.

    Thanks!


  • developer

    I think it's noteworthy that there's no 1:1 mapping between Android's name schema and VCard 3/4.

    Basically, there's a "display name" in Android which is mapped (by DAVdroid) to the VCard FN property, and the structured name (first name, last name etc.) which is mapped to the N property. As every VCard MUST have a FN, DAVdroid fills in the FN from the other fields, when necessary: this is why the first name suddenly appears as name.

    As the problem only occurs with Sony devices, I guess the Sony Contacts app does something special to the name fields, maybe an unusual format that's not covered by DAVdroid yet, or a bug in DAVdroid and/or the Sony contacts app.

    Unfortunately, I can't test it without a Sony device. You may have a look at the logs while playing around with different contacts with/without (first) name etc., always watching the transferred VCards. If DAVdroid makes a mistake, just let me know.

    However, I need a more exact description of the issue to see what's the cause and what can be done.



  • same here. I didn't had any problems with davdroid 0.7x, now with 0.8.1 and a fresh initialized/resettet ROM I trapped into.
    What is curious to me: some "new" contacts appear and are synced correct, nevertheless they are created on cloud or on phone. First I thought, initial sync was broken because I stopped it before it was finished. But as far I tested it with a fresh set up account, it is not that deterministic.
    Some details: https://www.evernote.com/l/AcCBRLiTdpRLP6qszTLjP2R8GEcRmC9mapw

    I have ~650 contacts in. 6 contacts are synced and visible on phone.

    during sync, logcat output shows sets like:

    D/davdroid.Collection(20205): Committing 3 operations ...
    D/SemcConfiguration(19075): Calling getBrandName(), the brand is null, read from overlay or local file.
    D/SemcConfiguration(19075): Calling getGroupVisible(), group_visible is false, read from overlay or local file.
    D/SemcConfiguration(19075): Calling getBrandName(), the brand is null, read from overlay or local file.
    D/SemcConfiguration(19075): Calling getGroupVisible(), group_visible is false, read from overlay or local file.
    D/SemcConfiguration(19075): Calling getBrandName(), the brand is null, read from overlay or local file.
    D/SemcConfiguration(19075): Calling getGroupVisible(), group_visible is false, read from overlay or local file.
    D/davdroid.Collection(20205): ... 0 row(s) affected
    

    where the set of SemcConfig {{Calling getBrandName()}} and {{Calling getGroupVisible()}} repeats as often davdroid tells how man operations are done: here: 3 times, this is different for each enty

    Sync finally finishes by:

    I/davdroid.SyncManager(12875): Fetching 0 updated remote resource(s)
    I/davdroid.SyncManager(12875): Removing non-dirty resources that are not present remotely anymore
    I/davdroid.SyncManager(12875): Sync complete, fetching new CTag
    I/davdroid.DavSyncAdapter(12875): Sync complete for com.android.contacts
    D/davdroid.DavSyncAdapter(12875): Closing httpClient
    

    Anyway: each sync seems to "process" ALL contacts again. This is inefficient but perhaps caused by davDroid itself doesn't find the contacts locally anymore.

    Note: What I can't change (anymore?!) in davdroid, is the option if VCard 4 should be used or not, it is force-enabled and can't be disabled. Perhaps this is related?

    Info: I'm on a Xperia SP C5303 Stock Android 4.3(.0)

    another log from a very short sync before I reset the contacts store and recreated the davdroid account respectivly if you just disable / reenable sync for contacts:

    I/ActivityManager(  857): Start proc at.bitfire.davdroid:sync for service at.bitfire.davdroid/.syncadapter.ContactsSyncAdapterService: pid=11899 uid=10218 gids={50218, 3003, 1028}
    I/ActivityManager(  857): Start proc android.process.acore for content provider com.android.providers.contacts/.SemcContactsProvider2: pid=11912 uid=10013 gids={50013, 1028}
    I/davdroid.DavSyncAdapter(11899): Performing sync for authority com.android.contacts
    D/davdroid.DavSyncAdapter(11899): Creating new DavHttpClient
    V/davdroid.TLS_SNI(11899): Setting allowed TLS protocols: TLSv1, TLSv1.1, TLSv1.2
    V/davdroid.TLS_SNI(11899): Setting allowed TLS ciphers: TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA
    I/ContactLocale(11912): AddressBook Labels [de_DE]: [, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, Sch, St, T, U, V, W, X, Y, Z, Α, Β, Γ, Δ, Ε, Ζ, Η, Θ, Ι, Κ, Λ, Μ, Ν, Ξ, Ο, Π, Ρ, Σ, Τ, Υ, Φ, Χ, Ψ, Ω, , А, Б, В, Г, Д, Е, Є, Ж, З, И, І, Ї, Й, К, Л, М, Н, О, П, Р, С, Т, У, Ф, Х, Ц, Ч, Ш, Щ, Ю, Я, , א, ב, ג, ד, ה, ו, ז, ח, ט, י, כ, ל, מ, נ, ס, ע, פ, צ, ק, ר, ש, ת, , ا, ب, ت, ث, ج, ح, خ, د, ذ, ر, ز, س, ش, ص, ض, ط, ظ, ع, غ, ف, ق, ك, ل, م, ن, ه, و, ي, , ก, ข, ฃ, ค, ฅ, ฆ, ง, จ, ฉ, ช, ซ, ฌ, ญ, ฎ, ฏ, ฐ, ฑ, ฒ, ณ, ด, ต, ถ, ท, ธ, น, บ, ป, ผ, ฝ, พ, ฟ, ภ, ม, ย, ร, ฤ, ล, ฦ, ว, ศ, ษ, ส, ห, ฬ, อ, ฮ, , ㄱ, ㄴ, ㄷ, ㄹ, ㅁ, ㅂ, ㅅ, ㅇ, ㅈ, ㅊ, ㅋ, ㅌ, ㅍ, ㅎ, , あ, か, さ, た, な, は, ま, や, ら, わ, #, ]
    V/davdroid.URIUtils(11899): Normalized URI https://owncloud.childno.de/remote.php/carddav/addressbooks/marcel/kontakte/ -> https://owncloud.childno.de/remote.php/carddav/addressbooks/marcel/kontakte/ assuming that it was an URI or path name
    D/davdroid.WebDavResource(11899): Using preemptive authentication (not compatible with Digest auth)
    I/davdroid.SyncManager(11899): Remotely removing 0 deleted resource(s) (if not changed)
    I/davdroid.SyncManager(11899): Uploading 0 new resource(s) (if not existing)
    I/davdroid.SyncManager(11899): Uploading 0 modified resource(s) (if not changed)
    I/dalvikvm(11899): Could not find method javax.xml.stream.XMLInputFactory.newInstance, referenced from method org.simpleframework.xml.stream.StreamProvider.<init>
    W/dalvikvm(11899): VFY: unable to resolve static method 3903: Ljavax/xml/stream/XMLInputFactory;.newInstance ()Ljavax/xml/stream/XMLInputFactory;
    D/dalvikvm(11899): VFY: replacing opcode 0x71 at 0x0003
    W/dalvikvm(11899): VFY: unable to find class referenced in signature (Ljavax/xml/stream/XMLEventReader;)
    W/dalvikvm(11899): VFY: unable to find class referenced in signature (Ljavax/xml/stream/XMLEventReader;)
    I/dalvikvm(11899): Could not find method javax.xml.stream.XMLInputFactory.createXMLEventReader, referenced from method org.simpleframework.xml.stream.StreamProvider.provide
    W/dalvikvm(11899): VFY: unable to resolve virtual method 3901: Ljavax/xml/stream/XMLInputFactory;.createXMLEventReader (Ljava/io/InputStream;)Ljavax/xml/stream/XMLEventReader;
    D/dalvikvm(11899): VFY: replacing opcode 0x6e at 0x0002
    I/dalvikvm(11899): Could not find method javax.xml.stream.XMLInputFactory.createXMLEventReader, referenced from method org.simpleframework.xml.stream.StreamProvider.provide
    W/dalvikvm(11899): VFY: unable to resolve virtual method 3902: Ljavax/xml/stream/XMLInputFactory;.createXMLEventReader (Ljava/io/Reader;)Ljavax/xml/stream/XMLEventReader;
    D/dalvikvm(11899): VFY: replacing opcode 0x6e at 0x0002
    I/sysmon-tsens_tz_sensor3(  314): sensor_work - read value = 360
    I/sysmon-tsens_tz_sensor0(  314): sensor_work - read value = 390
    V/davdroid.URIUtils(11899): Normalized URI /remote.php/carddav/addressbooks/marcel/kontakte/ -> /remote.php/carddav/addressbooks/marcel/kontakte/ assuming that it was an URI or path name
    D/davdroid.WebDavResource(11899): Processing multi-status element: https://owncloud.childno.de/remote.php/carddav/addressbooks/marcel/kontakte/
    D/davdroid.SyncManager(11899): Last local CTag = 1437376174; current remote CTag = 1437376174
    I/davdroid.SyncManager(11899): No local changes and CTags match, no need to sync
    I/davdroid.DavSyncAdapter(11899): Sync complete for com.android.contacts
    D/davdroid.DavSyncAdapter(11899): Closing httpClient
    

    :notebook_with_decorative_cover: more details and log-links will be attached to the evernote
    I hope this helps so far, if anything is missing or I can support, just let me know.

    I will compile https://github.com/android/platform_packages_apps_contacts branch jb-release contacts app for retesting

    ~Marcel



  • I have some more info as well...

    When I add a contact using Gmail account, the "Name" field (the one causing problems) is not shown... only "First name" and "Last name" are shown, and the rest are optional. That's maybe where the interaction between DavDroid and Sony Contacts app is failing!

    I have in my phone four different "contact providers" (however it's called) when I add a contact:

    • Phone contact (device only)
    • Google contact
    • Exchange activesync contact
    • Davdroid contact

    All other providers show "First Name" and "Last Name" fields by default (and very few alternatives), and it's only DavDroid showing "Name" and "Phonetic xxx" as defaults fields, and that's where the problem (at least my problem) comes from.

    Is there a way to tell DavDroid to show "First Name" and "Last name" fields as default like the other providers? Or are those defaults specified sistem-wide and DavDroid is not picking it up?

    Does VCARD4 have anything to do with it? I was not able to disable it!

    Thanks!



  • So I think our both issues differ. Other Contact Apps are working the same as my Sony Contacts list - for me - for my problem. All of them use the internal, stock Android 4.3 ContactStorage. Davdroid itself is an app and provides accounts and mightbe a Provider in a technical matter, but he doesn't store the contacts item itself. That's - how far I understood this - is not how Android is working where you have a core system storage manager and different Provider might use this.
    So the issue of wrongly / differently displayed items is caused by how each contact app interprets / reads the "vcard" data stored. @naevtamarkus Manipulating incoming data is not what I expect from DavDroid. What about then outgoing syncs? This has to be fixed at the root - where do the wrongly saved incoming vcards come from, e.g. owncloud, so it's an owncloud issue.

    But back to topic: I'm quite not sure if this is a third party issue? Reading my own logs it seems to be a somewhay incompatibility between the remote server's vcard data provider and davdroid. But then it's strange @nijel is able to "workaround" using a different Contact manager?

    So perhaps we might sum up what has been discussed so far:

    • original problem: only Sony contact app doesn't show items
      • workaround: use AOSP (or another) Contact manager app
    • derived problem on a Sony device where not all issues are synced but new ones are
      • not able to workaround using a different contact manager app
    • related issues:
      • FullName auto-split / storage / escaped specialchars
      • VCard 4 option can't be switched off in DavDroid 0.8.1

  • developer

    All other providers show "First Name" and "Last Name" fields by default (and very few alternatives), and it's only DavDroid showing "Name" and "Phonetic xxx" as defaults fields, and that's where the problem (at least my problem) comes from.

    On my phones (and all devices I have tested DAVdroid yet), the Contacts app shows First Name and Last Name as expected. It reads the possible fields from https://github.com/bitfireAT/davdroid/blob/master/app/src/main/res/xml/contacts.xml. This file is quite undocumented.

    Is there a way to tell DavDroid to show "First Name" and "Last name" fields as default like the other providers? Or are those defaults specified sistem-wide and DavDroid is not picking it up?

    I guess this is related to the contacts.xml mentiond above, but I don't know if there's a way to expand the view by default. It shouldn't be important anyway, because when you enter "Firstname Lastname" the Contacts app should split the name and set the appropriate fields. You may nevertheless expand the view to have the first name / last name edit fields.

    Does VCARD4 have anything to do with it? I was not able to disable it!

    It's not a Setting, it just shows whether your server is VCard/4 capable. The GUI element is disabled and should be grayed out to indicate that it can't be changed, but for some reason it's not grayed out.

    So, is there any issue left for DAVdroid or can I close this?



  • I think the "First Name"/"Last Name" behaivour is annoying, although I understand it's not very critical. May I try to fix this and send a pull request?



  • @rfc2822 I don't think there is a general agreement on your statement that this is "absolutely" not related to DavDroid. From all the providers there are (I tried four) only DavDroid gives you "Name" as an option where all the others give you First/Last name... so there is probably something that makes DavDroid different and this "feature" has a bad interaction with Sony devices.

    I am not saying there is a problem "per se" in DavDroid, and I honestly know very little about its internals to make a strong assertion about anything. But for (us) Sony users this is an annoying problem.

    Does any of the developers own a Sony device? Or anyone with expertise on this code? I would like to ask you to "Let The Community Work" and see what the options are. I would be very happy to try acceptable (by DavDroid devel's standards) proposals in my phone.

    Thanks!


  • developer

    I think the "First Name"/"Last Name" behaivour is annoying, although I understand it's not very critical. May I try to fix this and send a pull request?

    Of course, I guess you will have to play around with the (undocumented) EditSchema in contacts.xml. Or would you have another approach?

    From all the providers there are (I tried four) only DavDroid gives you "Name" as an option where all the others give you First/Last name...

    I just tried the built-in Exchange account (with outlook.com) and it indeed shows "First name" and "Last name" by default.

    Which did you try? Any of them open-source so that we can have a look?



  • Of course, I guess you will have to play around with the (undocumented) EditSchema in contacts.xml. Or would you have another approach?

    I guess you mean compiling the APK with different values in contacts.xml... but that's not very sustainable, isn't it? I mean, even if I succeeded (wouldn't know where to start from) I would have to change that on every new version. Or are you willing to accept pull-requests on such file?

    I just tried the built-in Exchange account (with outlook.com) and it indeed shows "First name" and "Last name" by default.
    Which did you try? Any of them open-source so that we can have a look?

    • Phone contact (device only)
    • Google contact
    • Exchange activesync contact
    • Davdroid contact

    I don't think any other is really open source... save, maybe, for the first one. There might be open implementations in AOSP or CM


  • developer

    I guess you mean compiling the APK with different values in contacts.xml... but that's not very sustainable, isn't it? I mean, even if I succeeded (wouldn't know where to start from) I would have to change that on every new version. Or are you willing to accept pull-requests on such file?

    Of course, if this change doesn't break anything (would have to be tested very well, because every single change breaks n vendors with m Android versions (n,m ∈ ℕ)).

    Phone contact (device only)
    Exchange activesync contact

    The contacts schemes should be available for at least these two ones, but I didn't find anything on the fly. It may also be that there's no possibility to change the display order because it may be bound to Android-internal account types by hard code (as the group feature is).



  • Somehow: it fixes itself since DavDroid 0.8.2 is installed and I synced back twice from Mac and Triggered Sync on Phone ... but perhaps: the items on Server somehow got deleted and they are all "new" now to the sync adapter ... Can someone verify this: export all contacts from server, sync empty list, add / reimport all contacts, re-sync -> contacts are resurrected on phone? fullstop, sorry... during Android DavDroid Sync was running, I went into the Sony Contacts list and many of my contacts appear, once it has been finished syncing, most of them are gone.. What the h***



  • I just encountered the same behavior as @childnode described: During sync all the contacts are populated in the stock Sony contacts app. You can actually watch them appear as they are synced. Once the sync finishes most of the contacts are removed. An alternative app (True Contacts) just shows the exact same contacts as the Sony app.

    Sony Z3C 5.1.1, DAVdroid 0.8.3, ownCloud 8.1.1

    edit: Same problem on my Nexus 7 2013 tablet. So in my case it's not Sony after all. And if I interrupt the sync manually the contacts stay there.
    At the end of the sync I get this: "Removing entries that are not present remotely anymore (retaining xxx entries)" I have exactly xxx contacts in ownCloud.
    Everyone may want to have a look at this solution: https://github.com/bitfireAT/davdroid/issues/578


  • admin

    Yes, you have to fix your owncloud as described in #578 (only change of one line is needed)

    Alternatively you can wait until OC releases the already submitted patch that fixes this issue!


Log in to reply
 

Looks like your connection to Bitfire App Forums was lost, please wait while we try to reconnect.