regression in DAVx5 2.6.1, 403 when PUTting calendar entry

infinity0

I recently upgraded to 2.6.1-ose and now calendar inserts (to apple calendarserver 9.2, on Debian) are no longer working with error code 403 (Client Forbidden). The server access logs show:

127.0.0.1 - infinity0 [XXX] "PUT /calendars/__uids__/XXX/calendar/XXX.ics HTTP/1.1" 403 335 "-" "DAVx5/2.6.1-ose (2019/11/23; dav4jvm; okhttp/3.12.6) Android/9" i=1 or=1 t=82.0 cl=693 err=valid-calendar-data

Previously with 2.6 it was all fine:

127.0.0.1 - infinity0 [XXX] "PUT /calendars/__uids__/XXX/calendar/XXX.ics HTTP/1.1" 201 0 "-" "DAVx5/2.6-ose (2019/10/28; dav4jvm; okhttp/3.12.6) Android/9" i=3 or=1 t=80.7 cl=689

rfc2822

Hi,

Please provide steps to reproduce, Web server and CalDAV/CardDAV server logs taken while the problem occurs, DAVx⁵ debug info (DAVx⁵ / Settings / Debug info) and verbose DAVx⁵ logs etc as described in [READ BEFORE POSTING] What’s required to diagnose a problem.

infinity0

Hi, steps to reproduce for me is (1) install 2.6.1 and (2) insert a calendar entry and try to synchronise, against an apple calendarserver. I don’t think I have any non-default settings, at least for caldav. The account is a “login with URL and username” type of account.

I have already downgraded back to 2.6 to get my account working again, and would prefer not to have to re-upgrade and re-downgrade because it takes a while to set up the accounts. (On android a direct downgrade is not possible, you have to uninstall and reinstall.)

The server log error err=valid-calendar-data suggests that DAVx5 2.6.1 is sending some data that’s different than in version 2.6, and it is this data that apple calendarserver does not like. I guess there have not been that many major changes from 2.6 to 2.6.1 in this area, so hopefully the bug can already be deduced without me having to reinstall 2.6.1 and provide more detailed error logs?

rfc2822

@infinity0 said in regression in DAVx5 2.6.1, 403 when PUTting calendar entry:

I guess there have not been that many major changes from 2.6 to 2.6.1 in this area, so hopefully the bug can already be deduced without me having to reinstall 2.6.1 and provide more detailed error logs?

Without logs, this is all a guessing game. I’ll try to reproduce the scenario, but it will be much work (dig out the Calendarserver VM, check the version, set up DAVx5, try it out…) and with logs I could probably just see the reason without any additional work.

Do you use email reminders for events? Which event did you try to upload?

infinity0

I don’t use email reminders, and the event details were pretty ordinary, with no special Unicode characters or anything like that.

I will try to find some time to produce some more detailed logs in the next few days.

rfc2822

Thanks. I have just tried to test with calendarserver, but it doesn’t start anymore and instead I get this problem (after having fixed another one).

Or can you maybe provide a temporary (Internet-accessible) test account?

infinity0

Which problem do you mean specifically? That thread is long and has a lot of separate issues. If you are talking about “calendarserver_upgrade is not working”, the fix is in message #50.

Alternatively you could try reinstalling a whole new calendarserver instead of upgrading - that thread is also nearly 4 years old.

rfc2822

@infinity0 Yes, I just got it running…

rfc2822

Can reproduce the problem:

<error xmlns='DAV:'>
  <valid-calendar-data xmlns='urn:ietf:params:xml:ns:caldav'/>
  <error-description xmlns='http://twistedmatrix.com/xml_namespace/dav/'><![CDATA[Calendar data had unfixable problems:
  [VEVENT] Missing or too many required property: DTSTAMP]]></error-description>
</error>

Seems that 2.6.1 doesn’t send a DTSTAMP anymore for whatever reason.

So thanks for the report, no additional information needed.

Bug-Ninja

I am experiencing the same problem, regardless of the calendar app I use. It seems DTSTAMP is missing when trying to update/add an appointment to the server. Server gives no problems on other devices (laptop with thunderbird and tbsync).

Debug info has been anonymized

--- BEGIN DEBUG INFO ---
SYNCHRONIZATION INFO
Account name: bugninja@xxx.com
Authority: com.android.calendar
 
HTTP REQUEST:
Request{method=PUT, url=https://calendar.xxx.com:8443/calendars/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/calendar/64313217-35f5-4765-bb3e-2dc84d914d10.ics, tags={}}
BEGIN:VCALENDAR 
VERSION:2.0 
PRODID:+//IDN bitfire.at//DAVx5/2.6.1-ose ical4j/2.2.6 
BEGIN:VEVENT 
UID:64313217-35f5-4765-bb3e-2dc84d914d10 
SUMMARY:Testphone
DESCRIPTION:Blub 
DTSTART;TZID=Europe/Amsterdam:20191125T180000 
DTEND;TZID=Europe/Amsterdam:20191125T190000 
STATUS:CONFIRMED 
BEGIN:VALARM 
TRIGGER:-PT10M 
ACTION:DISPLAY 
DESCRIPTION:Testphone
END:VALARM 
END:VEVENT 
BEGIN:VTIMEZONE 
TZID:Europe/Amsterdam 
TZURL:http://tzurl.org/zoneinfo/Europe/Amsterdam 
X-LIC-LOCATION:Europe/Amsterdam 
BEGIN:DAYLIGHT 
TZOFFSETFROM:+0100 
TZOFFSETTO:+0200 
TZNAME:CEST 
DTSTART:19810329T020000 
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU 
END:DAYLIGHT 
BEGIN:STANDARD 
TZOFFSETFROM:+0200 
TZOFFSETTO:+0100 
TZNAME:CET 
DTSTART:19961027T030000 
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU 
END:STANDARD 
BEGIN:STANDARD 
TZOFFSETFROM:+001932 
TZOFFSETTO:+001932 
TZNAME:AMT 
DTSTART:18350101T004028 
RDATE:18350101T004028 
END:STANDARD 
BEGIN:DAYLIGHT 
TZOFFSETFROM:+001932 
TZOFFSETTO:+011932 
TZNAME:NST 
DTSTART:19160501T004028 
RDATE:19160501T004028 
RDATE:19170416T024028 
RDATE:19180401T024028 
RDATE:19190407T024028 
RDATE:19200405T024028 
RDATE:19210404T024028 
RDATE:19220326T024028 
RDATE:19230601T024028 
RDATE:19240330T024028 
RDATE:19250605T024028 
RDATE:19260515T024028 
RDATE:19270515T024028 
RDATE:19280515T024028 
RDATE:19290515T024028 
RDATE:19300515T024028 
RDATE:19310515T024028 
RDATE:19320522T024028 
RDATE:19330515T024028 
RDATE:19340515T024028 
RDATE:19350515T024028 
RDATE:19360515T024028 
RDATE:19370522T024028 
END:DAYLIGHT 
BEGIN:STANDARD 
TZOFFSETFROM:+011932 
TZOFFSETTO:+001932 
TZNAME:AMT 
DTSTART:19161001T004028 
RDATE:19161001T004028 
RDATE:19170917T034028 
RDATE:19180930T034028 
RDATE:19190929T034028 
RDATE:19200927T034028 
RDATE:19210926T034028 
RDATE:19221008T034028 
RDATE:19231007T034028 
RDATE:19241005T034028 
RDATE:19251004T034028 
RDATE:19261003T034028 
RDATE:19271002T034028 
RDATE:19281007T034028 
RDATE:19291006T034028 
RDATE:19301005T034028 
RDATE:19311004T034028 
RDATE:19321002T034028 
RDATE:19331008T034028 
RDATE:19341007T034028 
RDATE:19351006T034028 
RDATE:19361004T034028 
END:STANDARD 
BEGIN:DAYLIGHT 
TZOFFSETFROM:+011932 
TZOFFSETTO:+0120 
TZNAME:+0120 
DTSTART:19370701T004000 
RDATE:19370701T004000 
END:DAYLIGHT 
BEGIN:STANDARD 
TZOFFSETFROM:+0120 
TZOFFSETTO:+0020 
TZNAME:+0020 
DTSTART:19371003T034000 
RDATE:19371003T034000 
RDATE:19381002T034000 
RDATE:19391008T034000 
END:STANDARD 
BEGIN:DAYLIGHT 
TZOFFSETFROM:+0020 
TZOFFSETTO:+0120 
TZNAME:+0120 
DTSTART:19380515T024000 
RDATE:19380515T024000 
RDATE:19390515T024000 
END:DAYLIGHT 
BEGIN:DAYLIGHT 
TZOFFSETFROM:+0020 
TZOFFSETTO:+0200 
TZNAME:CEST 
DTSTART:19400516T004000 
RDATE:19400516T004000 
END:DAYLIGHT 
BEGIN:STANDARD 
TZOFFSETFROM:+0200 
TZOFFSETTO:+0100 
TZNAME:CET 
DTSTART:19421102T030000 
RDATE:19421102T030000 
RDATE:19431004T030000 
RDATE:19441002T030000 
RDATE:19450916T030000 
RDATE:19770925T030000 
RDATE:19781001T030000 
RDATE:19790930T030000 
RDATE:19800928T030000 
RDATE:19810927T030000 
RDATE:19820926T030000 
RDATE:19830925T030000 
RDATE:19840930T030000 
RDATE:19850929T030000 
RDATE:19860928T030000 
RDATE:19870927T030000 
RDATE:19880925T030000 
RDATE:19890924T030000 
RDATE:19900930T030000 
RDATE:19910929T030000 
RDATE:19920927T030000 
RDATE:19930926T030000 
RDATE:19940925T030000 
RDATE:19950924T030000 
END:STANDARD 
BEGIN:DAYLIGHT 
TZOFFSETFROM:+0100 
TZOFFSETTO:+0200 
TZNAME:CEST 
DTSTART:19430329T020000 
RDATE:19430329T020000 
RDATE:19440403T020000 
RDATE:19450402T020000 
RDATE:19770403T020000 
RDATE:19780402T020000 
RDATE:19790401T020000 
RDATE:19800406T020000 
END:DAYLIGHT 
BEGIN:STANDARD 
TZOFFSETFROM:+0100 
TZOFFSETTO:+0100 
TZNAME:CET 
DTSTART:19770101T000000 
RDATE:19770101T000000 
END:STANDARD 
END:VTIMEZONE 
END:VCALENDAR 

 
HTTP RESPONSE:
Response{protocol=h2, code=403, message=, url=https://calendar.xxx.com:8443/calendars/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/calendar/64313217-35f5-4765-bb3e-2dc84d914d10.ics}

 
  
 Calendar data had unfixable problems:
 [VEVENT] Missing or too many required property: DTSTAMP 


 
LOCAL RESOURCE:
LocalEvent=[eTag=null, fileName=64313217-35f5-4765-bb3e-2dc84d914d10.ics, flags=0, weAreOrganizer=true, calendar=at.bitfire.davdroid.resource.LocalCalendar@f552574, event=Event=[alarms=[BEGIN:VALARM 
TRIGGER:-PT10M 
ACTION:DISPLAY 
DESCRIPTION:Testphone
END:VALARM 
], attendees=[], classification=null, color=null, description=Blub, dtEnd=DTEND;TZID=Europe/Amsterdam:20191125T190000, dtStart=DTSTART;TZID=Europe/Amsterdam:20191125T180000, duration=null, exDates=[], exRule=null, exceptions=[], lastModified=null, location=null, opaque=true, organizer=null, rDates=[], rRule=null, recurrenceId=null, status=STATUS:CONFIRMED, summary=Testphone, unknownProperties=[], sequence=0, uid=64313217-35f5-4765-bb3e-2dc84d914d10, shadow$_klass_=class at.bitfire.ical4android.Event, shadow$_monitor_=0], id=42, shadow$_klass_=class at.bitfire.davdroid.resource.LocalEvent, shadow$_monitor_=0]
 
REMOTE RESOURCE:
https://calendar.xxx.com:8443/calendars/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/calendar/64313217-35f5-4765-bb3e-2dc84d914d10.ics
 
EXCEPTION:
at.bitfire.dav4jvm.exception.ForbiddenException: HTTP 403 
at at.bitfire.dav4jvm.DavResource.checkStatus(DavResource.kt:354)
at at.bitfire.dav4jvm.DavResource.checkStatus(DavResource.kt:338)
at at.bitfire.dav4jvm.DavResource.put(DavResource.kt:244)
at at.bitfire.davdroid.syncadapter.SyncManager$uploadDirty$2$1.invoke(SyncManager.kt:329)
at at.bitfire.davdroid.syncadapter.SyncManager$uploadDirty$2$1.invoke(SyncManager.kt:54)
at at.bitfire.davdroid.syncadapter.SyncManager.useRemote(SyncManager.kt:812)
at at.bitfire.davdroid.syncadapter.SyncManager$uploadDirty$2.invoke(SyncManager.kt:316)
at at.bitfire.davdroid.syncadapter.SyncManager$uploadDirty$2.invoke(SyncManager.kt:54)
at at.bitfire.davdroid.syncadapter.SyncManager.useLocal(SyncManager.kt:798)
at at.bitfire.davdroid.syncadapter.SyncManager.uploadDirty(SyncManager.kt:312)
at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1.invoke(SyncManager.kt:127)
at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1.invoke(SyncManager.kt:54)
at at.bitfire.davdroid.syncadapter.SyncManager.unwrapExceptions(SyncManager.kt:838)
at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.kt:113)
at at.bitfire.davdroid.syncadapter.CalendarsSyncAdapterService$CalendarsSyncAdapter.sync(CalendarsSyncAdapterService.kt:57)
at at.bitfire.davdroid.syncadapter.SyncAdapterService$SyncAdapter.onPerformSync(SyncAdapterService.kt:74)
at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:321)
 
SOFTWARE INFORMATION
* at.bitfire.davdroid 2.6.1-ose (313) from com.google.android.packageinstaller
* org.dmfs.tasks 1.2.3 (77600) from com.google.android.packageinstaller
* com.android.providers.contacts 8.1.0 (27)
* com.android.providers.calendar 8.1.0 (27)
* com.android.contacts 1.7.21 (10721)
* com.simplemobiletools.calendar.pro 6.7.1 (165) from com.google.android.packageinstaller
* ws.xsoh.etar 1.0.19 (21) from com.google.android.packageinstaller
 
CONNECTIVITY (at the moment)
- [ Transports: WIFI Capabilities: NOT_METERED&INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN&VALIDATED&FOREGROUND LinkUpBandwidth>=1048576Kbps LinkDnBandwidth>=1048576Kbps SignalStrength: -55]
 
CONFIGURATION
Power saving disabled: yes
android.permission.READ_CONTACTS: granted
android.permission.WRITE_CONTACTS: granted
android.permission.READ_CALENDAR: granted
android.permission.WRITE_CALENDAR: granted
org.dmfs.permission.READ_TASKS: granted
org.dmfs.permission.WRITE_TASKS: granted
android.permission.ACCESS_COARSE_LOCATION: denied
System-wide synchronization: automatically
Account: bugninja@xxx.com
 Address book sync. interval: 60 min
 Calendar sync. interval: 60 min
 OpenTasks sync. interval: 60 min
 WiFi only: false
 [CardDAV] Contact group method: GROUP_VCARDS
 [CalDAV] Time range (past days): 90
 Manage calendar colors: true
 Use event colors: false
Address book account: addressbook (bugninja@xxx.com 8A)
 Main account: Account {name=bugninja@xxx.com, type=bitfire.at.davdroid}
 URL: https://calendar.xxx.com:8443/addressbooks/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/addressbook/
 Sync automatically: true
 
SQLITE DUMP
android_metadata
| locale |
| nl_NL |
----------
room_master_table
| id | identity_hash |
| 42 | a7554d1ebc5ecfe0c7aefe271485d63c |
----------
service
| id | accountName | type | principal |
| 1 | bugninja@freedom.nl | carddav | https://calendar.xxx.com:8443/principals/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/ |
| 2 | bugninja@xxx.com | caldav | https://calendar.xxx.com:8443/principals/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/ |
----------
sqlite_sequence
| name | seq |
| service | 2 |
| homeset | 2 |
| collection | 2 |
----------
homeset
| id | serviceId | url | privBind | displayName |
| 1 | 2 | https://calendar.xxx.com:8443/calendars/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/ | 1 | bugninja@xxx.com |
| 2 | 1 | https://calendar.xxx.com:8443/addressbooks/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/ | 1 | webninja@freedom.nl |
----------
collection
| id | serviceId | type | url | privWriteContent | privUnbind | forceReadOnly | displayName | description | color | timezone | supportsVEVENT | supportsVTODO | supportsVJOURNAL | source | sync |
| 1 | 2 | CALENDAR | https://calendar.xxx.com:8443/calendars/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/calendar/ | 1 | 1 | 0 | calendar |  | -72703 |  | 1 | 1 | 0 |  | 1 |
| 2 | 1 | ADDRESS_BOOK | https://calendar.xxx.com:8443/addressbooks/__uids__/EF9A3AAB-3D4B-48BA-85A5-8AB188F22572/addressbook/ | 1 | 1 | 0 |  |  |  |  |  |  |  |  | 1 |
----------
 
SYSTEM INFORMATION
Android version: 8.1.0 (lineage_bullhead-userdebug 8.1.0 OPM7.181205.001 6558e33ae4)
Device: LGE Nexus 5X (bullhead)
 
--- END DEBUG INFO ---

rfc2822

We will release a fix soon.

rfc2822

Should be fixed with 2.6.1.1. Please confirm whether its working for you

Bug-Ninja

@rfc2822 thanks a lot! Not seeing this version yet in F-droid, but will let you know once I have installed it.

infinity0

Thanks a lot! I too am waiting for f-droid.

rfc2822

2.6.1.1 is now in F-Droid (you may have to refresh the repository).

infinity0

Works great, thanks!

Bug-Ninja

I confirm!

rfc2822

Thanks for the information. So this is now solved