Duplicated contacts with Baïkal 0.2.6



  • The steps I followed are:

    1. Installed Baïkal 0.2.6 and setup one user with one addressbook.
    2. Installed DAVdroid on my phone and setup my DAVdroid contacts account.
    3. Exported all contacts to internal storage.
    4. Erased Contacts Storage.
    5. Imported contacts from internal storage into the DAVdroid account.

    Everything seemed to be working, the contacts appeared on the Baïkal server and I could see changes being synced back and forth.
    Then I started getting sync errors on my phone and noticed that the number of contacts Baïkal reported was 20x the number of contacts I started with.
    I know Android automatically merges contacts so I'm not sure if the 20x contacts only exist on the Baïkal server or if they exist on the phone also. I tried exporting all contacts to the SD card again and it did not show duplicate contacts.


  • developer

    Tried exactly as you said, but with only 3 contacts and Baïkal 0.2.6. No sync-back or sync errors here.

    How many contacts did you import? Which version of DAVdroid and Baikal did you use? Please test again with 0.3.6 which will soon be released. Do you know how to get logcat information for DAVdroid, or at least HTTP server logs? I need more information to locate the problem as I can't reproduce it here.



  • Hello, I do have the same issue with davdroid (was on 0.3.8 and still in 0.4.0) and baikal (0.2.6).

    Each time I manually synchronize the contacts on my phone, they are duplicated on the baikal server. Now from around 140 real contacts, almost 1000 are present on the baikal server.

    The apache2 logs show only the "PUT" request coming from the phone:

    [17/Nov/2013:10:08:45 +0100] 192.168.200.1 TLSv1 RC4-MD5 "PUT /card.php/addressbooks/nroosen/default/2acef77b-a9f8-48ef-9e77-dcd6bf3557cb.vcf HTTP/1.1" 292

    What's the best (easiest) option to get some davdroid logs on the phone?

    Thanks.


  • developer

    Do you know how to get logcat output? Is your device rooted? I will need all logcat entries tagged with "davdroid"



  • Well so far I'm downloading the Android SDK to get the adb.exe, then
    hopefully I'll be able to get the logs through the USB connection.

    I only hope it's working with an NON-rooted device ...

    On 17/11/2013 2:56 PM, rfc2822 wrote:

    Do you know how to get logcat output? Is your device rooted? I will
    need all logcat entries tagged with "davdroid"

    — Reply to this email directly or view it on GitHub
    https://github.com/rfc2822/davdroid/issues/51#issuecomment-28648843.


  • developer

    Yes adb works without rooting, but apps on the device require root (because logs may contain private content that shouldn't be read by other apps)



  • All righty, here are the logs (well I did some sorting, so hopefully
    it'll be enough):

    11-17 15:56:28.688 13122 13177 W davdroid.SNISocketFactory: No SNI
    support below Android 4.2!

    11-17 15:56:29.758 13122 13177 W SingleClientConnManager: Invalid use of
    SingleClientConnManager: connection still allocated.

    11-17 15:56:29.758 13122 13177 W SingleClientConnManager: Make sure to
    release the connection before allocating another one.

    11-17 15:57:03.738 13122 13177 E davdroid.SyncManager: Couldn't create
    entity for adding: net.fortuna.ical4j.model.ValidationException:
    Property [FN] must be specified once

    11-17 15:57:03.868 13122 13177 E davdroid.SyncManager: Couldn't create
    entity for adding: net.fortuna.ical4j.model.ValidationException:
    Property [FN] must be specified once

    11-17 15:57:03.868 13122 13177 I davdroid.LocalCollection: Committing
    294 operations

    11-17 15:57:03.968 12938 12948 E CursorWindow: Could not allocate
    CursorWindow
    '/data/user/0/com.android.providers.contacts/databases/contacts2.db' of
    size 2097152 due to error -12.

    11-17 15:57:03.968 12938 12948 W ContactsDatabaseHelper:
    invalidateAllCache: [ContactsDatabaseHelper]

    11-17 15:57:03.968 12938 12948 W ContactsDatabaseHelper:
    invalidateAllCache: [ProfileDatabaseHelper]

    11-17 15:57:03.968 12938 12948 E JavaBinder: *** Uncaught remote
    exception! (Exceptions are not yet supported across processes.)

    11-17 15:57:03.968 12938 12948 E JavaBinder:
    android.database.CursorWindowAllocationException: Cursor window
    allocation of 2048 kb failed. # Open Cursors=856 (# cursors opened by
    pid 13122=856)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.database.CursorWindow.<init>(CursorWindow.java:104)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.database.AbstractWindowedCursor.clearOrCreateWindow(AbstractWindowedCursor.java:198)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:139)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:133)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.database.AbstractCursor.moveToPosition(AbstractCursor.java:196)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.database.AbstractCursor.moveToNext(AbstractCursor.java:244)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    com.android.providers.contacts.ContactsProvider2.updateRawContacts(ContactsProvider2.java:4161)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    com.android.providers.contacts.ContactsProvider2.updateInTransaction(ContactsProvider2.java:3828)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    com.android.providers.contacts.AbstractContactsProvider.update(AbstractContactsProvider.java:143)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    com.android.providers.contacts.ContactsProvider2.update(ContactsProvider2.java:2027)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.content.ContentProviderOperation.apply(ContentProviderOperation.java:225)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    com.android.providers.contacts.AbstractContactsProvider.applyBatch(AbstractContactsProvider.java:214)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    com.android.providers.contacts.ContactsProvider2.applyBatch(ContactsProvider2.java:2146)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.content.ContentProvider$Transport.applyBatch(ContentProvider.java:222)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.content.ContentProviderNative.onTransact(ContentProviderNative.java:178)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    android.os.Binder.execTransact(Binder.java:367)

    11-17 15:57:03.968 12938 12948 E JavaBinder: at
    dalvik.system.NativeStart.run(Native Method)

    11-17 15:57:03.968 12938 13117 D ContactsProvider: Query
    URI:content://com.android.contacts/raw_contacts?account_type=bitfire.at.davdroid&account_name=nroosen%40werk35.nikolaii.net%2Fcard.php&caller_is_syncadapter=true

    11-17 15:57:03.978 12938 13117 E CursorWindow: Could not allocate
    CursorWindow
    '/data/user/0/com.android.providers.contacts/databases/contacts2.db' of
    size 2097152 due to error -12.

    On 17/11/2013 3:41 PM, rfc2822 wrote:

    Yes adb works without rooting, but apps on the device require root
    (because logs may contain private content that shouldn't be read by
    other apps)


    Reply to this email directly or view it on GitHub
    https://github.com/rfc2822/davdroid/issues/51#issuecomment-28649625.



  • So after a long list of those java exception errors, I got the following:

    11-17 15:57:18.048 13122 13177 D davdroid.WebDavResource: Received multistatus response: <?xml version="1.0" encoding="utf-8"?>

    11-17 15:57:18.048 13122 13177 D davdroid.WebDavResource: <d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:card="urn:ietf:params:xml:ns:carddav"><d:response><d:href>/card.php/addressbooks/nroosen/default/</d:href><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/">1181</x3:getctag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><d:getetag/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/ab5525fa-ddc3-455c-98f7-40e401ceb85d.vcf</d:href><d:propstat><d:prop><d:getetag>"c6c52440e667a9973d483bf45e65a4c2"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/292c1a62-a995-4381-94cf-bf487d04e255.vcf</d:href><d:propstat><d:prop><d:getetag>"01daa53e5772b458d4472945289f37da"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/e90668e4-20c0-43cf-95e3-aa141474638f.vcf</d:href><d:propstat><d:prop><d:getetag>"313af2353dc0dd6eb2881a76a3813cc0"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/02289bc0-6e53-46df-b849-4ba0db73aa06.vcf</d:href><d:propstat><d:prop><d:getetag>"7c98631f01c4a9e4ee1530b30dc96671"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/840dbf0a-10de-451d-a105-98737c06c60e.vcf</d:href><d:propstat><d:prop><d:getetag>"ee5f147ccbfc6a20fc373b87d282a5b7"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/e6a6187b-daf1-4d48-a9fc-14807234af4e.vcf</d:href><d:propstat><d:prop><d:getetag>"ac5e555c6974d9fc945da9675c7998d2"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/0837d9c8-6561-4a5a-8fb1-37c512da7b49.vcf</d:href><d:propstat><d:prop><d:getetag>"5597c5f7f8f09d204c0415aebfc4899b"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/de4ff013-659d-4db4-aea4-82dd227e5d4b.vcf</d:href><d:propstat><d:prop><d:getetag>"05fc489f7fe191fccb79dcc02c14e646"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response><d:response><d:href>/card.php/addressbooks/nroosen/default/dd5c9a1f-32ea-4bd1-8cf0-eae8d3486da9.vcf</d:href><d:propstat><d:prop><d:getetag>"9468f9bf75b5c5e19ab28626d7a8cf91"</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat><d:propstat><d:prop><x3:getctag xmlns:x3="http://calendarserver.org/ns/"/></d:prop><d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:r

    11-17 15:57:18.048 13122 13177 D davdroid.WebDavResource: Processing multi-status element: https://werk35.nikolaii.net:8443/card.php/addressbooks/nroosen/default/

    11-17 15:57:18.048 13122 13177 D davdroid.WebDavResource: Processing multi-status element: https://werk35.nikolaii.net:8443/card.php/addressbooks/nroosen/default/ab5525fa-ddc3-455c-98f7-40e401ceb85d.vcf

    [ ... ]

    11-17 15:57:19.958 13122 13177 D davdroid.WebDavResource: Processing multi-status element: https://werk35.nikolaii.net:8443/card.php/addressbooks/nroosen/default/32369e33-5913-44b3-94d1-436706e290e8.vcf

    11-17 15:57:20.018 12938 13178 D ContactsProvider: Query URI:content://com.android.contacts/raw_contacts?account_type=bitfire.at.davdroid&account_name=nroosen%40werk35.nikolaii.net%2Fcard.php&caller_is_syncadapter=true

    11-17 15:57:20.028 12938 13178 E CursorWindow: Could not allocate CursorWindow '/data/user/0/com.android.providers.contacts/databases/contacts2.db' of size 2097152 due to error -12.

    11-17 15:57:20.028 12938 13178 E JavaBinder: *** Uncaught remote exception! (Exceptions are not yet supported across processes.)

    11-17 15:57:20.028 12938 13178 E JavaBinder: android.database.CursorWindowAllocationException: Cursor window allocation of 2048 kb failed. # Open Cursors=856 (# cursors opened by pid 13122=856)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at android.database.CursorWindow.<init>(CursorWindow.java:104)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at android.database.AbstractWindowedCursor.clearOrCreateWindow(AbstractWindowedCursor.java:198)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:139)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:133)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at android.database.CursorToBulkCursorAdaptor.getBulkCursorDescriptor(CursorToBulkCursorAdaptor.java:143)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at android.content.ContentProviderNative.onTransact(ContentProviderNative.java:121)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at android.os.Binder.execTransact(Binder.java:367)

    11-17 15:57:20.028 12938 13178 E JavaBinder: at dalvik.system.NativeStart.run(Native Method)


  • developer

    2 MB... seems like you have a LOT of contacts or very big data stored with them. Is that correct?



  • Well at least on my phone there are only 139 contacts, with nothing but
    phone numbers and e-mail addresses in it.

    But now on the server side, since all contacts are duplicated each time
    I do a manual synchronization, I do have around 1100 contacts.

    On 17/11/2013 5:48 PM, rfc2822 wrote:

    2 MB... seems like you have a LOT of contacts or very big data stored
    with them. Is that correct?


    Reply to this email directly or view it on GitHub
    https://github.com/rfc2822/davdroid/issues/51#issuecomment-28652190.


  • developer

    Did you try with a clean / empty account? The last error is caused because 1100 contacts are too many.



  • So I deleted all my contacts (after having backed them up onto the
    memory phone in one single vcf file) on the phone AND on the server.

    I created a new account on the server.
    Then imported my contacts back on the phone.
    Then synchronized the davdroid account.
    Then checked on the Baikal dashboard: only 86 contacts out of 139 where
    imported.
    After the sync, when I open the contact list, it is empty.

    I still do have those errors in the debug logs:

    11-17 19:45:28.577 12938:13179 E/CursorWindow ]

    Could not allocate CursorWindow
    '/data/user/0/com.android.providers.contacts/databases/contacts2.db' of
    size 2097152 due to error -12.

    [ 11-17 19:45:28.587 12938:13179 E/JavaBinder ]

    *** Uncaught remote exception! (Exceptions are not yet supported across
    processes.)

    android.database.CursorWindowAllocationException: Cursor window
    allocation of 2048 kb failed. # Open Cursors=856 (# cursors opened by
    pid 19167=856)

    at android.database.CursorWindow.<init>(CursorWindow.java:104)
    
    at
    

    android.database.AbstractWindowedCursor.clearOrCreateWindow(AbstractWindowedCursor.java:198)

    at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:139)
    
    at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:133)
    
    at
    

    android.database.CursorToBulkCursorAdaptor.getBulkCursorDescriptor(CursorToBulkCursorAdaptor.java:143)

    at
    

    android.content.ContentProviderNative.onTransact(ContentProviderNative.java:121)

    at android.os.Binder.execTransact(Binder.java:367)
    
    at dalvik.system.NativeStart.run(Native Method)
    

    Maybe the local phone database is corrupted, is there a way to clean it ?

    On 17/11/2013 7:11 PM, rfc2822 wrote:

    Did you try with a clean / empty account? The last error is caused
    because 1100 contacts are too many.


    Reply to this email directly or view it on GitHub
    https://github.com/rfc2822/davdroid/issues/51#issuecomment-28658885.



  • Having said that, I decided to test the creation of a new contact. Then the contact application died. I would like to attach the logfile (too many lines to paste them here), but is there a way to upload a text file on Github?

    I don't know if it's relevant, but I'm using a Motorola Razri, which host an Intel CPU.


  • developer

    Maybe you can upload the logs to gist.github.com and then link it?


  • developer

    DAVdroid 0.5 comes with VCard 3 support as required by the CardDAV specs and some minor tweaks. Please test again (don't forget to delete your account and create it again with DAVdroid 0.5) and if it still doesn't work, please comment here again.



  • I tested this with 0.5.2-alpha and it is still duplicating contacts.



  • I have a similar duplicate contacts issue using DAVdroid 0.5.2-alpha and OwnCloud 6.0.

    • I install OwnCloud 6.0 and DAVdroid 0.5.2-alpha.
    • On my phone, I start with no contacts and configure DAVdroid to sync with OwnCloud for CardDAV.
    • I import a list of contacts.
    • Everything looks correct.
    • After some amount of time, I check back in OwnCloud and the number of contacts has increased, and increases, and keeps increasing.

    Are there others who have OwnCloud as a CardDAV server and don't have this problem? Alternately, what CardDAV servers are successfully working?

    One more question for @rfc2822 . How many contacts is too many? I have something like 500. Also, most of them have pictures. This means a contacts.vcf file is around 4M in size. Do I need to be worried?



  • These are the steps I followed with 0.5.2-alpha and Baïkal 0.2.6:

    1. Exported all contacts to a vcf file.
    2. Deleted old DAVdroid account, verified that contacts disappeared from contacts app.
    3. Tried to delete the default address-book for my user with Baïkal's web admin interface but that choked and never finished loading the page and the address-book remained. Instead I deleted all rows from the cards table in the sqlite database, verified that the web admin interface showed no contacts in my address-book.
    4. Added new DAVdroid account to phone.
    5. Imported contacts from vcf file.
    6. Watched the contact count on the web admin interface grow past 500 contacts while the phone said it was still syncing. I have just under 500 contacts on the phone.


  • As I wrote above, I'm having the same contacts duplication bug using OwnCloud. For the phone, I've tested on a Galaxy S3 and using an emulated phone on my computer. Both behave the same (i.e., badly).

    Both Baikal and OwnCloud appear to use a SabreDAV back end. So possibly we're seeing the same underlying problem. (But I am just speculating.)

    Is this issue a good place for me to comment about my problems using OwnCloud, or would it be better to open another?


  • developer

    Please try again with DAVdroid 0.5.4 (uninstall/delete all accounts, then install DAVdroid 0.5.4 and add accounts again). There have been many improvements related to a larger number of contacts/events as well as performance optimisations.


Log in to reply
 

Looks like your connection to Bitfire App Forums was lost, please wait while we try to reconnect.