CALDAV sync fails with Sync thread cancelled! Interrupting sync" and "Couldn't check/increase sequence"

[dd1303]

Hello,

the initial sync of my CALDAV calender (5547 entries, oldest from 1997) between davx5 and radicale (or time20 as well ) fails with:

code_text
```2020-12-22 09:01:07 15558 [syncadapter.SyncAdapterService] Sync thread cancelled! Interrupting sync
2020-12-22 09:01:07 2 [CustomCertService] CustomCertService destroyed
2020-12-22 09:01:07 15547 [syncadapter.CalendarsSyncAdapterService] Couldn't sync calendars
EXCEPTION java.lang.InterruptedException
        at com.mikepenz.aboutlibraries.R$style.runBlocking(Unknown Source:23)
        at at.bitfire.davdroid.syncadapter.SyncManager.uploadDirty(SyncManager.kt:2)
        at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1.invoke(SyncManager.kt:8)
        at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1.invoke(SyncManager.kt:1)
        at at.bitfire.davdroid.syncadapter.SyncManager.unwrapExceptions(SyncManager.kt:1)
        at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.kt:5)
        at at.bitfire.davdroid.syncadapter.CalendarsSyncAdapterService$CalendarsSyncAdapter.sync(CalendarsSyncAdapterService.kt:14)
        at at.bitfire.davdroid.syncadapter.SyncAdapterService$SyncAdapter.onPerformSync(SyncAdapterService.kt:13)
        at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:334)```
code_text

And 1162 of these “Couldn’t check/increase sequence”:

code_text
```2020-12-22 09:02:45 15556 [resource.LocalCalendar] Couldn't check/increase sequence
EXCEPTION android.os.DeadObjectException
        at android.os.BinderProxy.transactNative(Native Method)
        at android.os.BinderProxy.transact(BinderProxy.java:511)
        at android.content.ContentProviderProxy.query(ContentProviderNative.java:421)
        at android.content.ContentProviderClient.query(ContentProviderClient.java:195)
        at android.content.ContentProviderClient.query(ContentProviderClient.java:177)
        at android.content.ContentProviderClient.query(ContentProviderClient.java:167)
        at at.bitfire.ical4android.AndroidEvent.getEvent(AndroidEvent.kt:5)
        at at.bitfire.davdroid.resource.LocalCalendar.findDirty(LocalCalendar.kt:3)
        at at.bitfire.davdroid.syncadapter.SyncManager$uploadDirty$1.invokeSuspend(SyncManager.kt:2)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:3)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:18)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)```
code_text

Icount 4538 lines of the kind “PARAMETER #0 …” which seems to be created for each calendar entry.

This is how I tried it:

I setup caldav/carddav-server on ubuntu and added that account to davx5. Synced with MPE from Outlook to davx5. Synced with davx5 to caldav/carddav-server.

Sync of the CARDDAV contacts (1148 entries) is fine.

The failure for CALDAV showed up before (around February 2020?) on Samsung S2 before I switched to Xiaomi.

I currently use:

• Windows 10 with Outlook
• list itemMPE MyPhoneExplorer 1.8.15
• davx5 3.3.7-ose
• Xiaomi Redmi Note 8 Pro (MIUI 1.0.2, Android 10 QP1A.190711.020)
• Lubuntu 18.04
• radicale 3.0.6 or
• tine20 2.0 2020.11.4 (business trial version)

My data:

• CARDDAV: 1148 contacts
• list itemCALDAV: 5547 calendar entries, oldest from 1997

When I reduced the calendar in davx5-Android to 1987 entries (1700 days before today …) it synced.

So I’m not sure whether this is a size limitation or whether there is a bad cadavx5-debug (26).zip lender entry.

Best regards,
Dietrich

[dd1303]

Hi,

is it possible that I encounter something similar as described here?

Best regards,
Dietrich

[devvv4ever]

@dd1303 I don’t think that has something to with that in the linked thread.

I don’t know what causes that Couldn’t check/increase sequence message, but it must be something very grotesque (possibly it has to do with attendies and the sequence number of an event). If you can export your whole data set and send it to us we can try to reproduce that.

You can also try to leave the past time event limit option empty. Then DAVx5 will use a different method for transmitting the data (named collection sync).

Best regards,
Bernhard

[dd1303]

Hi Bernhard, I’m quite sure that I left the option “past time limit” empty in order to get all my (past) calendar entries synced. But I can’t check as I re-installed davx5 completely. I took the chance of my free time during the holidays and split my calendar into a big one, with old “static” entries (dated until 31.12.2019, ca. 5100 entries) and a small one (entries from 1.1.2020 until future, ca. 400 entries). I’m now syncing only the small one, which works fine. I still have an ICS export of the failing calendar with me an it is a very nice offer that you are willing to investigate it. But it contains sensitive data and I hesitate to send it away “as it is”. Is there a way to make it “neutral”? I could try to replace the subject, location and description with xxxx. Would that be an option for you?

Best regards,
Dietrich

[devvv4ever]

@dd1303 It would be an option, yes.

[dd1303]

@devvv4ever attached please find the offending calendar as ICS. I created it with CIE (Calendar Import-Export) and replaced the text for summary, description and location by xxx and also created new UIDs a I saw double values created by CIE. I imported xxx.ICS successfully with CIE to a local Android calendar.
dd-caldav-2-xxx.ics

[devvv4ever]

@dd1303 I’ve tried it now, but due to the anonymization radicale can’t import it… Is it possible to create a test account where the original ics is being used? Or alternatively any test account where the error occurs? Otherwise we can’t have a deeper look into it…

[dd1303]

@devvv4ever I can create a test account on my radicale server and try to do the initial sync from davx5 to that account with the original ICS. How would you take a look then? Should I send you a davx5-debug.zip again?

[dd1303]

@dd1303 18:xx created a new radicale calendar called “xxx”
18:xx created a new davx5 account called “xxx” and assigned the calendar xxx to it
18:xx executed a davx5 sync with the empty calendar, after that the calendar is visible for the Android calendar app
18:26 executed a successful sync with one test entry “test termin” created with my calendar app in calendar xxx
18:54 on Android with Calendar Import-Export loaded the 5548 entries from dd-caldav-2-xxx.ics into calendar xxx
19:00 sync started in davx5 for xxx
19:00:13 radicale output stopped without any sign of processing any calendar entries
19:02:xx davx5 orange activity bar stopped
19:02:12 the known error “[syncadapter.SyncAdapterService] Sync thread cancelled! Interrupting sync” shows up in dav5-log.txt

radicale.log
davx5-debug (35).zip
I’m able to handle the anonymized ICS file and can re-create the same error as described in my first posting:

2020-12-29 19:02:12 7318 [syncadapter.SyncAdapterService] Sync thread cancelled! Interrupting sync
2020-12-29 19:02:12 2 [CustomCertService] CustomCertService destroyed
2020-12-29 19:02:12 7351 [syncadapter.CalendarsSyncAdapterService] Couldn't sync calendars

[dd1303]

@dd1303 Oups, I had to many xxx in the continuation lines of dd-caldav-2-xxx.ics. Calendar Import-Export didn’t mind, but when I imported it with Lightning into radicale I saw errors about dtend and dtstart. Attached please find the corrected version dd-caldav-2-xxx2.ics and hopefully this allows you to import it to radicale.
dd-caldav-2-xxx2.ics

[rfc2822]

@dd1303 Does the exception appear again when you kill DAVx5 in the task manager and then sync again?

[rfc2822]

EXCEPTION java.lang.InterruptedException
        at com.mikepenz.aboutlibraries.R$style.runBlocking(Unknown Source:23)
        at at.bitfire.davdroid.syncadapter.SyncManager.uploadDirty(SyncManager.kt:2)

It’s some very weird problem that I couldn’t reproduce or understand yet. Probably it’s related to threaded coroutines. Once upon a time, the sync algorithm gets mixed with “com.mikepenz.aboutlibraries.R$style.runBlocking” (which is in no way related to the sync algorithm), and then the sync algorithm code behaves in an undefined way and doesn’t catch the exceptions anymore (although the code is the same and try/catch is there)…

This is why I couldn’t reproduce the problem with your data (thanks btw). It’s not related to the specific event data, but to some other very strange phenomenon.

The InterruptedException is normal; Android throws it when synchronization runs too long. But sync should just continue when it’s called again.

[dd1303]

@rfc2822 I executed a sync, failed, killed davx5 from the Android apps settings, started davx5, tried the sync again and failed with the same exception.

[dd1303]

@dd1303 Hi - do you still follow on this issue and want to fix it? In that case I am willing to support, i.e give it some other tries if required. Otherwise it’s fine with me to close it. I can continue with my workaround: one big static calendar (entries from past until 31.12.2019); plus a smaller one (entries since 1.1.2020 towards future) which is syncing fine.
Best regards,
Dietrich