Error 500 when syncing calendar/tasks (DAVDroid 0.9.13, OwnDrive 8.2.2 (stable))

[breversa]

Hi,

I'm getting an error 500 when trying to syncing my default calendar/task list from my OwnDrive account on my Samsung Galaxy Nexus running CyanogenMod 12.1/Android 5.1.1). Contacts sync fine.

Please note that there is also a Contacts' birthday calendar that I do NOT sync.

Here are the logs :

Calendar :

[info ] Preparing synchronization
[info ] Querying capabilities
[debug] Received <response> for https://my.owndrive.com/remote.php/caldav/calendars/[username]/[calname]/, status: null, properties: [calendar-color(http://apple.com/ns/ical/): CalendarColor(color=-65536),
getctag(http://calendarserver.org/ns/): GetCTag(cTag=47),
displayname(DAV:): DisplayName(displayName=Calendrier Owndrive)]
[info ] Setting calendar name and color (if available)
[info ] Processing locally deleted entries
[info ] Locally preparing dirty entries
[info ] Uploading dirty entries
[info ] Checking sync state
[info ] Listing local entries
[debug] Found local resource: f03f9ae0-0fb4-47f6-8e6d-77a203cc7d6e.ics
[debug] Found local resource: c4ae5d62-8d32-45e2-8731-52345298dee8.ics
[debug] Found local resource: 6e7d5c50-fc07-468e-923d-9af1208bec8d.ics
[debug] Found local resource: 2de67d06-dd26-4a0c-a457-4177bb81db1f.ics
[info ] Listing remote entries
[error] HTTP/DAV Exception during sync - EXCEPTION:
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
	at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:281)
	at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:286)
	at at.bitfire.dav4android.DavCalendar.calendarQuery(DavCalendar.java:79)
	at         at.bitfire.davdroid.syncadapter.CalendarSyncManager.listRemote(CalendarSyncManager.java:117)
at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.java:172)
at     at.bitfire.davdroid.syncadapter.CalendarsSyncAdapterService$SyncAdapter.onPerformSync(CalendarsSyncAdapterService.java:58)
at     android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:281)
at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:286)
at at.bitfire.dav4android.DavCalendar.calendarQuery(DavCalendar.java:79)
at     at.bitfire.davdroid.syncadapter.CalendarSyncManager.listRemote(CalendarSyncManager.java:117)
at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.java:172)
at     at.bitfire.davdroid.syncadapter.CalendarsSyncAdapterService$SyncAdapter.onPerformSync(CalendarsSyncAdapterService.java:58)
at     android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)

Tasks :

[info ] Preparing synchronization
[info ] Querying capabilities
[debug] Received <response> for https://my.owndrive.com/remote.php/caldav/calendars/[username]/[calname]/, status: null, properties: [calendar-color(http://apple.com/ns/ical/): CalendarColor(color=-65536),
displayname(DAV:): DisplayName(displayName=Calendrier Owndrive),
getctag(http://calendarserver.org/ns/): GetCTag(cTag=47)]
[info ] Setting task list name and color (if available)
[info ] Processing locally deleted entries
[info ] Locally preparing dirty entries
[info ] Uploading dirty entries
[info ] Checking sync state
[info ] Listing local entries
[info ] Listing remote entries
[error] HTTP/DAV Exception during sync - EXCEPTION:
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:281)
at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:286)
at at.bitfire.dav4android.DavCalendar.calendarQuery(DavCalendar.java:79)
at     at.bitfire.davdroid.syncadapter.TasksSyncManager.listRemote(TasksSyncManager.java:115)
at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.java:172)
at     at.bitfire.davdroid.syncadapter.TasksSyncAdapterService$SyncAdapter.onPerformSync(TasksSyncAdapterService.java:63)
at     android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:281)
at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:286)
at at.bitfire.dav4android.DavCalendar.calendarQuery(DavCalendar.java:79)
at     at.bitfire.davdroid.syncadapter.TasksSyncManager.listRemote(TasksSyncManager.java:115)
at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.java:172)
at     at.bitfire.davdroid.syncadapter.TasksSyncAdapterService$SyncAdapter.onPerformSync(TasksSyncAdapterService.java:63)
at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:259)

This behaviour may have started before version 0.9.13, and persists with version 1.0-beta1 (from F-Droid)

[rfc2822]

Hello @breversa,

[error] HTTP/DAV Exception during sync - EXCEPTION:
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error

As you can see, the server reports an internal error. This is a server problem.

[Topper]

Sorry RFC but I confirm this issue too.

I get ERROR 500 sync with my calendar since a few days. And all has been working ok for months before.

Details :

• My server is a web based ownCloud provider (openmailbox.org)
• All was working fine for months before
• Last thing I remember doing before error 500 was adding an emoji to a calendar event (don't know if it's relevent, maybe a DAVdroid didn't like it...)
• Calendar sync is impossible ever since
• Sync keeps working on my laptop with another client, with same credentials so it's a problem with DAVdroid
• Tried to reinstall the account/reinstall DAVDroid/Wipe cache/Upgrade to beta....Still getting error 500 !

This is crazy, it's like my phone has been blacklisted from the server now

Please help enclosed is the debug file from DAVDROID

0_1458771559975_error_log.html

[rfc2822]

Hello,

@Topper said:

Sorry RFC but I confirm this issue too.

I didn't say that this problem is not existing, I'm just saying that it's most probably a server issue. After all, the error message is "500 internal server error".

I get ERROR 500 sync with my calendar since a few days. And all has been working ok for months before.

So, it seems there's a server problem now.

• Sync keeps working on my laptop with another client, with same credentials so it's a problem with DAVdroid

This is not a valid conclusion. Other clients use other types of requests to get calendars. For instance, they may fetch the list of entries with PROPFIND instead of the (better) REPORT calendar-query, and they may not use the (better)calendar-multiget and fetch all entries separately. To have a comparison, we would have to know which specific requests are actually working and which are not.

• Tried to reinstall the account/reinstall DAVDroid/Wipe cache/Upgrade to beta....Still getting error 500 !

Yes, because it's most probably a server issue.

Please help enclosed is the debug file from DAVDROID

0_1458771559975_error_log.html

The HTTP request seems fine for me, and the response indicates a server error.

I'm open for better suggestions and hints, but I can't see any DAVdroid problem here.

Did you already try to get in contact with your provider? If you do so, please always attach the logs, they're quite clear.

[breversa]

Hello Topper and RFC,

Topper :
Thank you for your input : I have exactly the same issue/symptoms as you. Thanks for confirming I'm not the only one.

RFC :
I'll have to check again, but I believe my other Android device has no problem syncing calendar/tasks… and uses DAVDroid too (I don't use any other sync client on Android devices).

I haven't contacted OwnDrive regarding this issue yet, but I will.

[Topper]

Thanks for following up.

I'm not technical enough to dig in the guts of my laptop calendar client (korganiser KDE 5, linux), but well at least it works whatever the request it uses.
I agree that 500 is a server failure but shouldn't DAVdroid consider this issue in terms of compatibility and use a more failsafe method ? We're talking about two regular owncloud service providers here...

Because asking for a server configuration change is really out of reach of the average end user

Anyway, I found an (ugly) workaround. I backed up my calendar as ics file, created a new one and reimported it. Works back again. No error 500 so far.

Finally and above all, great app, and thanks for your work !

[breversa]

Topper,

So you're using KDE too ? That's another common point ! (Aren't you french also ?)

Anyway, can you tell me how you exported your calendar as ics file ? What app did you use ? The standard Android agenda app does not offer such an operation…

[Topper]

@breversa

Oui je suis français et utilisateur Archlinux/KDE
Pour l'export je suis simplement allé dans mon espace web owncloud, calendrier, et il y a une option pour exporter le calendrier en ics là dedans Ensuite je reimporte aussi depuis owncloud et c'est fini.

[breversa]

I knew you were a fellow countryman ! ^^

I gave your workaround a try, but no luck : I get a "file not found" when trying to download my calendar as .ics file. If I create a new calendar, no problem, so I guess there's something fishy about that particular calendar. I emailed tech support on that and will try again after their reply. Merci camarade !

[Topper]

"File not found" ? That's weird.

When I go to my calendar in my owncloud web account, I click to export the calendar, then it goes as an ics file in the owncloud root directory.
Then I reimport it by clicking it. And that does the trick.

[breversa]

@Topper
Apparently only that one calendar is buggy. Any other calendar works fine. I emailed support about it and they'll do something about it next week.

[breversa]

My issue was caused by a broken event in the calendar. It did not show up, so I had my provider support delete it for me.

Once deleted, I could download, rename and resync my calendar.

(Now how do I change this topic's title to add "[SOLVED]" ?)

[rfc2822]

@breversa Thanks for telling us the solution of your problem. I have flagged the topic as solved.

[breversa]

But come to think of it : isn't there room for improvement to avoid such behavior (or maybe just more useful messages) with broken events ?

[rfc2822]

@breversa +1 The server should return a better message than "500 Internal Server Error" without further details.

[breversa]

Sorry about the thread necromancy, but I have some news :
After checking with my OwnCloud (8.2.2) provider, I confirm that creating a calendar event with emoji in the title (I'm using v1.2.2-ose) actually creates an event that doesn't properly end with 'END:VCALENDAR'.

Such events are invisible in the calendar web UI (since they don't have an end date, I guess ?) and hence cannot be deleted by end users such as myself.

Also, they break downloading the entire calendar file and thus sync with other devices.

I've had to contact my support service several times already to ask them to find and delete those events for me, which is a major drag for everyone and takes several days each time, during which I cannot sync events to my devices (I can sync from a device to the server, but no longer from the server to the devices).

I hence believe this is a breaking bug that should be addressed.

[rfc2822]

I can't reproduce that here, iCals with emojis are created and sent correctly.

Did you check the iCal on the server or in DAVdroid's verbose logs? Please always provide verbose logs.

[breversa]

@rfc2822
My bad, here are the error logs on my devices when trying to sync from the server :

SYNCHRONIZATION INFO
Synchronization phase: 7
Account name: ***
Authority: org.dmfs.tasks

HTTP REQUEST:
REPORT /remote.php/caldav/calendars/***/calendrierowndrive/
Depth: 1

<?xml version='1.0' encoding='UTF-8' ?><CAL:calendar-query xmlns="DAV:" xmlns:CAL="urn:ietf:params:xml:ns:caldav"><prop><getetag /></prop><CAL:filter><CAL:comp-filter name="VCALENDAR"><CAL:comp-filter name="VTODO" /></CAL:comp-filter></CAL:filter></CAL:calendar-query>

HTTP RESPONSE:
http/1.1 500 Internal Server Error
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: keep-alive
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *
Content-Type: application/xml; charset=utf-8
Date: Mon, 08 Aug 2016 22:24:00 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: Apache
Transfer-Encoding: chunked
X-Content-Type-Options: nosniff
X-Frame-Options: Sameorigin
X-Robots-Tag: none
X-Sabre-Version: 2.1.7
X-XSS-Protection: 1; mode=block

<?xml version="1.0" encoding="utf-8"?>[LF]
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">[LF]
  <s:sabredav-version>2.1.7</s:sabredav-version>[LF]
  <s:exception>Sabre\VObject\EofException</s:exception>[LF]
  <s:message>End of document reached prematurely</s:message>[LF]
</d:error>[LF]


EXCEPTION:
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
	at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:316)
	at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:289)
	at at.bitfire.dav4android.DavCalendar.calendarQuery(DavCalendar.java:88)
	at at.bitfire.davdroid.syncadapter.TasksSyncManager.listRemote(TasksSyncManager.java:108)
	at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.java:161)
	at at.bitfire.davdroid.syncadapter.TasksSyncAdapterService$SyncAdapter.onPerformSync(TasksSyncAdapterService.java:78)
	at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:272)

SOFTWARE INFORMATION
DAVdroid version: 1.2.2-ose (111) Thu Aug 04 15:39:02 GMT+02:00 2016
Installed from: org.fdroid.fdroid
JB Workaround installed: no

CONFIGURATION
System-wide synchronization: automatically
Account: ***
  Address book sync. interval: 240 min
  Calendar     sync. interval: 240 min
  OpenTasks    sync. interval: 240 min
  Preemptive auth: true
  WiFi only: false
  [CardDAV] Contact group method: CATEGORIES
            RFC 6868 encoding: true
  [CalDAV] Time range (past days): 90
           Manage calendar colors: false

SQLITE DUMP
android_metadata
	|  locale |
	|  fr_FR |
----------
settings
	|  setting | value |
----------
services
	|  _id | accountName | service | principal |
	|  5 | *** | carddav | https://my.owndrive.com/remote.php/carddav/principals/***/ |
	|  6 | *** | caldav | https://my.owndrive.com/remote.php/caldav/principals/***/ |
----------
sqlite_sequence
	|  name | seq |
	|  services | 6 |
	|  homesets | 9 |
	|  collections | 14 |
----------
homesets
	|  _id | serviceID | url |
	|  8 | 6 | https://my.owndrive.com/remote.php/caldav/calendars/***/ |
	|  9 | 5 | https://my.owndrive.com/remote.php/carddav/addressbooks/***/ |
----------
collections
	|  _id | serviceID | url | readOnly | displayName | description | color | timezone | supportsVEVENT | supportsVTODO | sync |
	|  11 | 6 | https://my.owndrive.com/remote.php/caldav/calendars/***/contact_birthdays/ | 0 | Anniversaires des contacts | <null> | -3355444 | <null> | 1 | 0 | 0 |
	|  12 | 6 | https://my.owndrive.com/remote.php/caldav/calendars/***/calendrierowndrive/ | 0 | Calendrier OwnDrive | <null> | -65536 | <null> | 1 | 1 | 1 |
	|  14 | 5 | https://my.owndrive.com/remote.php/carddav/addressbooks/***/4b65d2b6-cd67-47ec-aa1e-cce4fd6ea87b/ | 0 | Contacts OwnDrive | <null> | <null> | <null> | <null> | <null> | 1 |
----------

SYSTEM INFORMATION
Android version: 6.0.1 (cm_i9100-userdebug 6.0.1 MOB30R bbcf972f98 test-keys)
Device: Samsung GT-I9100 (GT-I9100)

SYNCHRONIZATION INFO
Synchronization phase: 7
Account name: ***
Authority: com.android.calendar

HTTP REQUEST:
REPORT /remote.php/caldav/calendars/***/calendrierowndrive/
Depth: 1

<?xml version='1.0' encoding='UTF-8' ?><CAL:calendar-query xmlns="DAV:" xmlns:CAL="urn:ietf:params:xml:ns:caldav"><prop><getetag /></prop><CAL:filter><CAL:comp-filter name="VCALENDAR"><CAL:comp-filter name="VEVENT"><CAL:time-range start="20160511T002354Z" /></CAL:comp-filter></CAL:comp-filter></CAL:filter></CAL:calendar-query>

HTTP RESPONSE:
http/1.1 500 Internal Server Error
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: keep-alive
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *
Content-Type: application/xml; charset=utf-8
Date: Mon, 08 Aug 2016 22:23:56 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: Apache
Transfer-Encoding: chunked
X-Content-Type-Options: nosniff
X-Frame-Options: Sameorigin
X-Robots-Tag: none
X-Sabre-Version: 2.1.7
X-XSS-Protection: 1; mode=block

<?xml version="1.0" encoding="utf-8"?>[LF]
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">[LF]
  <s:sabredav-version>2.1.7</s:sabredav-version>[LF]
  <s:exception>Sabre\VObject\EofException</s:exception>[LF]
  <s:message>End of document reached prematurely</s:message>[LF]
</d:error>[LF]


EXCEPTION:
at.bitfire.dav4android.exception.HttpException: 500 Internal Server Error
	at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:316)
	at at.bitfire.dav4android.DavResource.checkStatus(DavResource.java:289)
	at at.bitfire.dav4android.DavCalendar.calendarQuery(DavCalendar.java:88)
	at at.bitfire.davdroid.syncadapter.CalendarSyncManager.listRemote(CalendarSyncManager.java:126)
	at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.java:161)
	at at.bitfire.davdroid.syncadapter.CalendarsSyncAdapterService$SyncAdapter.onPerformSync(CalendarsSyncAdapterService.java:69)
	at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:272)

SOFTWARE INFORMATION
DAVdroid version: 1.2.2-ose (111) Thu Aug 04 15:39:02 GMT+02:00 2016
Installed from: org.fdroid.fdroid
JB Workaround installed: no

CONFIGURATION
System-wide synchronization: automatically
Account: ***
  Address book sync. interval: 240 min
  Calendar     sync. interval: 240 min
  OpenTasks    sync. interval: 240 min
  Preemptive auth: true
  WiFi only: false
  [CardDAV] Contact group method: CATEGORIES
            RFC 6868 encoding: true
  [CalDAV] Time range (past days): 90
           Manage calendar colors: false

SQLITE DUMP
android_metadata
	|  locale |
	|  fr_FR |
----------
settings
	|  setting | value |
----------
services
	|  _id | accountName | service | principal |
	|  5 | *** | carddav | https://my.owndrive.com/remote.php/carddav/principals/***/ |
	|  6 | *** | caldav | https://my.owndrive.com/remote.php/caldav/principals/***/ |
----------
sqlite_sequence
	|  name | seq |
	|  services | 6 |
	|  homesets | 9 |
	|  collections | 14 |
----------
homesets
	|  _id | serviceID | url |
	|  8 | 6 | https://my.owndrive.com/remote.php/caldav/calendars/***/ |
	|  9 | 5 | https://my.owndrive.com/remote.php/carddav/addressbooks/***/ |
----------
collections
	|  _id | serviceID | url | readOnly | displayName | description | color | timezone | supportsVEVENT | supportsVTODO | sync |
	|  11 | 6 | https://my.owndrive.com/remote.php/caldav/calendars/***/contact_birthdays/ | 0 | Anniversaires des contacts | <null> | -3355444 | <null> | 1 | 0 | 0 |
	|  12 | 6 | https://my.owndrive.com/remote.php/caldav/calendars/***/calendrierowndrive/ | 0 | Calendrier OwnDrive | <null> | -65536 | <null> | 1 | 1 | 1 |
	|  14 | 5 | https://my.owndrive.com/remote.php/carddav/addressbooks/***/4b65d2b6-cd67-47ec-aa1e-cce4fd6ea87b/ | 0 | Contacts OwnDrive | <null> | <null> | <null> | <null> | <null> | 1 |
----------

SYSTEM INFORMATION
Android version: 6.0.1 (cm_i9100-userdebug 6.0.1 MOB30R bbcf972f98 test-keys)
Device: Samsung GT-I9100 (GT-I9100)

Please note that I do NOT have access to the admin/system part of the server, only the user interface (otherwise I wouldn't need to bug the support service !).

Also, for what it's worth, I'm using Swiftkey Keyboard to type in emojis. I believe the problematic one represents those two cheering beer mugs :

[rfc2822]

@breversa There are not verbose logs, but the debug info. These only show the exceptions reported by the server. To get the verbose logs:

1. disable global sync on your Android device
2. enable verbose logs in DAVdroid
3. create an event with emoji on your device
4. manually synchronize with DAVdroid
5. disable verbose logs and re-enable global sync

The verbose logs will show the network traffic. In my tests, the VEVENT is uploaded correctly, so I'm quite sure it's an OwnCloud problem, probably caused by the limited Unicode implementation of the MySQL backend.

[breversa]

Sorry again for the misunderstanding.

Here are the logs (there were two files, so only one may be relevant) :

0_1470740117217_davdroid-9314-20160809-124100.txt

0_1470740135725_davdroid-6960-20160809-124100.txt

Thanks for the link. My technical german is a bit flaky, but my may very well be spot on the error cause.