Photo lost after new sync to owncloud



  • Owncloud 8.2.1, davdroid 0.9.1.2

    If I change a contact on the mobile phone which has a photo after the sync the photo is gone. It doesn't matter if the photo was initially added from the phone or from any other client. Changes on other clients (non android) do not show this effect.

    Is that a known issue or is that something new? If new can I provide something to help solve it?



  • I'm experiencing this as well. I previously had this setup working correctly.

    • Owncloud 8.2 server
    • Two computers with Thunderbird / Sogo connector.
    • Two Android phones with DavDroid 0.8.4.1

    I recently bought a new Android 5.1 phone and installed the latest DavDroid on it. 0.9.1.2 and the problem described above happens: Contact has photo. If any part of the contact is edited on the phone, when it syncs back to owncloud the photo is lost. This change propagates back to all other devices. The photo can be added again on Thunderbird or Owncloud and it will appear back on the Android 5.1 phone, so it appears to be a one-way problem.

    Possibly related: it seems to be impossible to add a photo to a contact on the phone: the normal edit mechanism is not available.



  • in regards to your possibley related: I can add photos to the contact, so the edit mechanism is there. It is just the sync causing issues. The rest of your description is like I have it, I just don't have thunderbird but Kontact.



  • New information ... just removed Davdroid 0.9.1.2 and tried with Davdroid 0.8.4.1. Same result, so we can rule out the DavDroid upgrade as a cause. So probably something to do with Android 5.1?



  • OK, finally, I deleted DavDroid and used CardDav (sorry) to sync with owncloud. No issues with losing photos there. So I guess that makes it something to do with DavDroid and Android Lollipop. Hope this helps.

    [Related Issue update: I can add a photo to a contact, only by getting the photo in my gallery and using "Set Picture As > Contact Photo". Still unable to add photo to contact in edit mode, so that appears unrelated. Sorry for the red herring.]


  • developer

    Hello,

    As far as I know,

    • OwnCloud advertises that it's VCard 4.0-capable, but has problems with VCard 4.0 data (including photos) in certain cases,
    • DAVdroid uses VCard 4.0 when the server advertises VCard 4.0,
    • CardDav-Sync ("CardDav") is not able to use VCard 4.0.

    So, in my opinion, this is

    1. an OwnCloud problem,
    2. it works with CardDav-Sync because it doesn't support the modern VCard 4.0 version.

    However, there will be no certainty without logs, which is the reason why there are verbose instructions in [Please read before posting] What's required to diagnose a problem.



  • @rfc2822 said:

    So, in my opinion, this is

    1. an OwnCloud problem,
    2. it works with CardDav-Sync because it doesn't support the modern VCard 4.0 version.

    However, there will be no certainty without logs, which is the reason why there are verbose instructions in [Please read before posting] What's required to diagnose a problem.

    I'm unable to post logs as I don't have DavDroid installed right now. Maybe I'll put it on an old phone and try. However these are my data points:

    • Owncloud 8.1, Davdroid 0.8, Android 4.2 works
    • Owncloud 8.1, Davdroid 0.8, Android 5.1 doesn't work
    • Owncloud 8.1, Davdroid 0.9, Android 5.1 doesn't work

    The server remained the same throughout, so I tend to draw a different conclusion: something about the Android 5.1 address book is different.


  • developer

    @plutocrat said:

    The server remained the same throughout, so I tend to draw a different conclusion: something about the Android 5.1 address book is different.

    I'm using DAVdroid with Android 5.1 myself, and it works without problems.

    However, without logs, that's all speculation and won't produce any useful insights.



  • Hi,

    I created some logs, they seem to be a bit odd.

    I created a contact. Added a photo and triggered the sync, the full log is http://pastebin.com/8LUzL4g7 The photo was as expected gone after the sync.

    SOFTWARE INFORMATION
    DAVdroid version: 0.9.1.2 (86) Sun Jan 03 15:39:07 MEZ 2016
    Installed from: APK (directly)
    JB Workaround installed: no
    
    SYSTEM INFORMATION
    Android version: 5.0 (LRX21T.G900FXXU1BOJ1)
    Device: Samsung SM-G900F (klte)
    

    Next I wanted to do it again to see if the same log is being created. I thought that the photo was there after the sync for this other test. So I checked on the owncloud server but there was no photo (didn't do any change on the server). I triggered the sync again. This time the photo was gone again on the mobile device. I am not sure what I did when I thought the photo was still there. What surprises me is that the log is completely different. This log is here http://pastebin.com/PMNhvCk5

    To avoid confusion: the first log contains 1 sync, the second log contains 2 syncs. The sync in the first log shows the issue, the second sync in the second log shows on the device exactly the same behavior. The first sync in the second log is a bit strange to me.

    Hope that helps, let me know if I can do anything else.

    Regards,
    K.


  • developer

    @Kalle What I can see in the logs:

    First log: there's a sync, but nothing has changed since the last sync. There is no upload and no download. I assume that you didn't disable global sync, so there was a sync immediately after you have changed the photo. Then you have triggered another sync (which then doesn't contain any uploads/downloads), and the last sync has been captured in the logs. So this log doesn't say anything. Did you disable global sync when taking the logs?

    Second log: A DAVdroid log file always contains only one synchronization (it will be overwritten by the next one). In this second log file, we can see

    1. DAVdroid querying address book info
    2. DAVdroid uploads a new VCard 4 ("Tesz") with photo using PUT to the resource /owncloud/remote.php/carddav/addressbooks/XXX/oc_YY_contacts/8371c8ee-48db-46a7-a48f-7337f8ac2d84.vcf (calculated from UID:8371c8ee-48db-46a7-a48f-7337f8ac2d84). You can see that it's a new (and not an updated) contact because
      • logs say "Uploading new record 8371c8ee-48db-46a7-a48f-7337f8ac2d84.vcf"
      • the PUT request has If-None-Match: *, i.e. the server must not overwrite an existing contact with this file name.
    3. However, and that's what's surprising, OwnCloud returns HTTP/1.1 412 Precondition failed. This means it doesn't accept the new VCard 8371c8ee-48db-46a7-a48f-7337f8ac2d84.vcf because there's already a contact with this file name. So, either the UUID is duplicate (which is very unlikely), or it has somehow become marked as "new" on your local device, or there's another reason I don't know.
    4. So, the upload of the "new" record has failed and was ignored to not overwrite existing data on the server.
    5. The rest of the logs is quite boring: DAVdroid lists the contents of the address book.
    6. The address book also contains 8371c8ee-48db-46a7-a48f-7337f8ac2d84.vcf (so it really seems to exist; it has been created on REV:2016-01-03T14:53:24+00:00), which is then downloaded and overwrites the "new" contact with the new photo, so that the old photo is restored.


  • Hi,

    maybe it was a create on the second log and not an update. I created a new log http://pastebin.com/raw/neSz3HWM This one looks rather like an update and the http response on the PUT request is a 204 no content. Though you see the photo data in the log file. Does that help any further?

    Thanks, K.


  • developer

    @Kalle said:

    maybe it was a create on the second log and not an update. I created a new log http://pastebin.com/raw/neSz3HWM This one looks rather like an update and the http response on the PUT request is a 204 no content. Though you see the photo data in the log file. Does that help any further?

    This log doesn't show any problems. The VCard with the PHOTO was uploaded successfully (204).

    So, this looks like an OwnCloud problem, as said above.



  • Hi,

    from an external point of view hard to believe that this is an owncloud bug. Sorry to be so frank. You say that davdroid uploads the pictures successfully to owncloud, owncloud is not showing it (which might be an owncloud bug), but it is being deleted (by davdroid!) from the Android device as well. If that is an owncloud bug, then owncloud would have to tell davdroid to delete the picture. If that is the reason, do you see in the log that owncloud is really telling davdroid to delete the picture?

    I am willing to debug this on my end to help improve owncloud/davdroid. In order to be able to debug it, can you please explain a little what is the trigger that davdroid deletes the image? Maybe you can give me even a nudge to the right class in your code that I can see the issue in the debugger.

    Thanks for your help and patience, it is highly appreciated.

    K.



  • Cross post to https://forum.owncloud.org/viewtopic.php?f=24&t=32392 to see if a collaboration might help to solve this.


  • developer

    @Kalle said:

    You say that davdroid uploads the pictures successfully to owncloud,

    Indeed.

    owncloud is not showing it (which might be an owncloud bug), but it is being deleted (by davdroid!) from the Android device as well. If that is an owncloud bug, then owncloud would have to tell davdroid to delete the picture. If that is the reason, do you see in the log that owncloud is really telling davdroid to delete the picture?

    Indeed. It's telling DAVdroid that the resource DAVdroid tries to upload is already existing on the server / has changed in the meanwhile (= conflict, see If-Match in Creating Address Object Resources). In this case, the server always wins.

    I am willing to debug this on my end to help improve owncloud/davdroid. In order to be able to debug it, can you please explain a little what is the trigger that davdroid deletes the image? Maybe you can give me even a nudge to the right class in your code that I can see the issue in the debugger.

    DAVdroid uploads the VCard with a pre-condition: "only if there's no existing VCard with this resource name" (If-None-Match: *). This pre-conditions fails, i.e. OwnCloud tells DAVdroid that there already is a resource with this name. So, DAVdroid doesn't overwrite it (and thus doesn't set the new photo), but downloads the resource from the server instead and uses this version.



  • Hello,

    I registered right now because I've been suffering from a similar problem for quite some time already, and I might have something to add.

    My setup :
    Samsung Galaxy Nexus (maguro) running CyanogenMod 12.1 (unofficial)/Android 5.1.1
    DAVdroid 0.9.1.3 (but the problem started several versions earlier)
    OwnCloud hosted by OwnDrive.com (8.2.2 Stable)

    I also have a Samsung Galaxy Tab 3 (running CyanogenMod 12.1 as well), and use OwnDrive's web interface to manage my contacts. Those do no pose any problem.

    SOFTWARE INFORMATION
    DAVdroid version: 0.9.1.3 (87) Sun Jan 03 08:04:57 GMT+01:00 2016
    Installed from: org.fdroid.fdroid
    JB Workaround installed: no
    
    CONFIGURATION
    System-wide synchronization: manually
    Account: breversa
      Address book sync. interval: manually
      Calendar     sync. interval: manually
      OpenTasks    sync. interval: 1440 min
    
    SYSTEM INFORMATION
    Android version: 5.1.1 (cm_maguro-userdebug 5.1.1 LMY48Y 59b31bc6e5 test-keys)
    Device: Samsung Galaxy Nexus (maguro)
    

    So, I have the same problem as kalle and plutocrat : adding/modifying a contact on my Galaxy Nexus causes the picture to disappear/be deleted in my OwnCloud/Drive account, then synced to my other devices.

    However, here's what what I found in the logcat :

    01-12 16:07:58.473 E/SQLiteLog(22926): (284) automatic index on sqlite_sq_49B9E920(STAT_DATA_ID)
    01-12 16:07:58.476 E/SQLiteLog(22926): (284) automatic index on sqlite_sq_42ADD9B8(STAT_DATA_ID)
    01-12 16:08:51.727 D/skia    (22926): --- SkImageDecoder::Factory returned null
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): Could not process photo for insert or update
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): java.io.IOException: Invalid image file
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.PhotoProcessor.process(PhotoProcessor.java:181)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.PhotoProcessor.<init>(PhotoProcessor.java:155)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.PhotoProcessor.<init>(PhotoProcessor.java:134)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.DataRowHandlerForPhoto.processPhoto(DataRowHandlerForPhoto.java:149)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.DataRowHandlerForPhoto.preProcessPhoto(DataRowHandlerForPhoto.java:111)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.DataRowHandlerForPhoto.insert(DataRowHandlerForPhoto.java:65)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.ContactsProvider2.insertData(ContactsProvider2.java:3090)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.ContactsProvider2.insertInTransaction(ContactsProvider2.java:2798)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.AbstractContactsProvider.insert(AbstractContactsProvider.java:136)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at com.android.providers.contacts.ContactsProvider2.insert(ContactsProvider2.java:2405)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at android.content.ContentProvider$Transport.insert(ContentProvider.java:239)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at android.content.ContentProviderNative.onTransact(ContentProviderNative.java:163)
    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): 	at android.os.Binder.execTransact(Binder.java:446)
    01-12 16:09:10.298 E/SQLiteLog(22926): (284) automatic index on sqlite_sq_4995AC40(STAT_DATA_ID)
    01-12 16:09:10.301 E/SQLiteLog(22926): (284) automatic index on sqlite_sq_423DB620(STAT_DATA_ID)
    

    Those lines do not appear when I add a picture to a contact, but when the contact gets synced (either manually or automatically). Actually, if I did'nt sync, the pictures would stay there, and those lines would not appear.

    Here's DAVdroid log for that sync session :

    [info ] Preparing synchronization
    [info ] Querying capabilities
    [debug] Received <response> for https://my.owndrive.com/remote.php/carddav/addressbooks/breversa/contacts/, status: null, properties: [getctag(http://calendarserver.org/ns/): GetCTag(cTag=1452551439),
    supported-address-data(urn:ietf:params:xml:ns:carddav): at.bitfire.dav4android.property.SupportedAddressData@188b1cfd]
    [info ] Server advertises VCard/4 support: true
    [info ] Processing locally deleted entries
    [info ] Locally preparing dirty entries
    [info ] Uploading dirty entries
    [info ] Uploading locally modified record foo.vcf
    [debug] Didn't receive new ETag after uploading, setting to null
    [info ] Checking sync state
    [info ] Manual sync, ignoring CTag
    [info ] Listing local entries
    [debug] Found local resource: foo.vcf
    ...
    [info ] Listing remote entries
    [debug] Received <response> for https://my.owndrive.com/remote.php/carddav/addressbooks/breversa/contacts/bar.vcf, status: null, properties: [getetag(DAV:): GetETag(eTag=897cd51b4083062f74847610d9b53fb5)]
    ...
    [debug] Found remote VCard: baz.vcf
    ...
    [info ] Comparing local/remote entries
    [info ] foo.vcf has been changed on server (current ETag=1ae98496bba5547436c930597a488f22, last known ETag=null)
    [info ] Downloading remote entries
    [info ] Downloading 1 contacts (10 at once)
    [info ] Downloading https://my.owndrive.com/remote.php/carddav/addressbooks/breversa/contacts/foo.vcf
    [info ] Updating foo.vcf in local address book
    [info ] Saving sync state
    [info ] Saving CTag=1452551439
    

  • developer

    01-12 16:08:51.736 E/DataRowHandlerForPhoto(22926): java.io.IOException: Invalid image file

    Seems that uploading a photo causes this – uploaded – photo to be invalid. It then disappears in the ownCloud Web Interface, and the invalid photo is also synchronized to other clients, which ignore the (invalid) photo, so that the photo is removed there, too.

    This seems to match the ownCloud Contacts issue that VCard 4 PHOTO uploads corrupt ownCloud's photo storage.

    See also https://github.com/owncloud/contacts/issues/1114 and the linked issues there.

    @breversa Do you use the latest ownCloud Contacts app version?



  • The only version info I have is "OwnDrive 8.2.2 (stable)". I don't have anything else, sorry. :-(

    I do however think that this disappearing-picture problem arose when I installed the first DAVdroid version that use VCard/4. I think it was fine with VCard/3, but I cannot confirm.


  • developer

    @breversa said:

    The only version info I have is "OwnDrive 8.2.2 (stable)". I don't have anything else, sorry. :-(

    Then I'd say it's a well-known – and already fixed – ownCloud Contacts app issue. If you can't do your self, please contact your hosting company to upgrade the Contacts app.

    Of course, if there's any indication that there's something wrong on DAVdroid's side, please let us know, but it should be backed up by some information what exactly is supposedly wrong.



  • Alright, I'll try and get in touch with OwnDrive and let you know if anything good comes out of it. Thanks you ! :-)


Log in to reply
 

Looks like your connection to Bitfire App Forums was lost, please wait while we try to reconnect.