DAVdroid can't handle events prior 1950

[flosch]

Dear all

I’m struggling with events prior 1950 not being rendered correctly on Android. It seems that DAVdroid is the problem here, as the very same calendar is rendered perfectly using aCalDAV. Following steps to reproduce:

• on the mobile phone created two events, the first on 25 August 1950, the second on 25 August 1949
• it doesn’t matter if these are recurring or single events
• it also doesn’t matter if these are all-day events or last for just a couple of hours, minutes, …
• synchronised with the CalDAV server
• both events are rendered correctly on the PC using Thunderbird/Lightning
• on the mobile phone, however, only that on 25 August 1950 is shown
• synchronising the very same calendar with aCalDAV, however, both events are now shown on the mobile phone

I didn’t determine the exact date but its somewhen between x.x.1949 and x.x.1950. This is the content of the first event, the second is identical but for the date and the summary:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:+//IDN bitfire.at//DAVdroid/1.7.1-ose ical4j/2.x
BEGIN:VEVENT
DTSTAMP:20170825T172303Z
UID:2d75036f-9f58-4a14-b35d-09881a34fa81
SUMMARY:test1
DTSTART;VALUE=DATE:19500825
DURATION:P1D
RRULE:FREQ=YEARLY;WKST=MO
STATUS:CONFIRMED
TRANSP:TRANSPARENT
END:VEVENT
END:VCALENDAR

Clearly, this is a bug in DAVdroid, possibly related to some integer overflow or whatever. It seems that DAVdroid can’t handle its own events (both events were initially created by DAVdroid, see PRODID above) whereas aCalDAV renders the very same events perfectly well. Events initially created in Thunderbird/Lightning are also affected.

Any solution/help is much appreciated. Thanks a lot in advance.

Cheers!

Florian

[rfc2822]

Hello,

I can’t reproduce this with our testing device. DAVdroid just inserts this event with these data:

PARAMETER #0 = mType: 1, mUri: content://com.android.calendar/events?account_name=test%40example.com&account_type=bitfire.at.davdroid&caller_is_syncadapter=true&account_name=test%40example.com&account_type=bitfire.at.davdroid&caller_is_syncadapter=true, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: eventTimezone=Etc/UTC title=test1 availability=1 eventStatus=1 allDay=1 hasAttendeeData=1 dtstart=-610761600000 calendar_id=1 duration=P1D rrule=FREQ=YEARLY;WKST=MO, mValuesBackReferences: null, mSelectionArgsBackReferences: null

and is shown in every year. If there’s a problem, I guess it’s with the calendar provider of your device which creates the actual events instances.

Note that Android is limited to dates after 1902: https://forums.bitfire.at/topic/1227/recurring-events-before-year-1902-are-not-shown-in-calendar/5

[flosch]

Hi rfc2822,

thanks a lot for testing. I am aware of the 1902 restriction which is clearly an Android limitation; and truely annoying, too. Don’t really understand the intention of restricting a calendar to a particular epoch, like life hadn’t existed before 1902. But that’s another topic.

I’ve never seen the structure of your data within the ics files on my CalDAV server. Am I guessing right, that this data is what DAVdroid sends to the CalDAV server and then the server actually creates the ics file? Anyhow, this doesn’t explain why DAVdroid can’t handle files of the structure I posted, whereas aCalDAV on Android and Thunderbird/Lightning on the PC can. If it were the calendar provider, the event would just never show up on Android. That’s why I think that there’s got to something to be fixed in DAVdroid.

Best of regards

Florian

[rfc2822]

@flosch said in DAVdroid can't handle events prior 1950:

I’ve never seen the structure of your data within the ics files on my CalDAV server. Am I guessing right, that this data is what DAVdroid sends to the CalDAV server and then the server actually creates the ics file?

No. What I have posted is the data as required by the Android Calendar Provider. DAVdroid parses the VEVENT from the ics file and converts it into data for Android calendar provider (see the image on https://davdroid.bitfire.at/faq/entry/system-integration/).

Anyhow, this doesn’t explain why DAVdroid can’t handle files of the structure I posted, whereas aCalDAV on Android and Thunderbird/Lightning on the PC can. There’s got to be something to fix this in DAVdroid.

It can handle it here. So I guess it’s a problem of your calendar provider. It’s indeed strange that aCalDAV works on your device, but of course I can’t give you any details because I can’t reproduce the problem here.

[flosch]

Thanks again for replying so quickly. As stated above, if aCalDAV and Thunderbird/Lightning can handle these events, I would assume that it’s not a problem with the calendar provider. That makes just no sense to me.

I understand perfecly well that you can’t help on a problem which you can’t reproduce. Is there any additional information that could help? Anything I could try instead? Any idea where the problem could be?

Edit: judging from your information on how the data is handled, I would then assume the problem is how DAVdroid converts the data for the Android calendar provider.

[rfc2822]

@flosch said in DAVdroid can't handle events prior 1950:

I understand perfecly well that you can’t help on a problem which you can’t reproduce. Is there any additional information that could help? Anything I could try instead? Any idea where the problem could be?

It would be necessary to reproduce the problem. Which device do you have? I suggest to post the debug info, as described in https://forums.bitfire.at/topic/749/read-before-posting-what-s-required-to-diagnose-a-problem/ It contains very important data like the device model and Android version, which is the reason why its should really be provided together with every problem report or question.

Edit: judging from your information on how the data is handled, I would then assume the problem is how DAVdroid converts the data for the Android calendar provider.

Why? I have posted the data which is generated by DAVdroid and it seems to be correct:

• dtstart=-610761600000: UNIX time -610761600 is Fri Aug 25 00:00:00 1950 UTC ✓
• duration=P1D seems to be OK for an all-day event ✓
• rrule=FREQ=YEARLY;WKST=MO directly taken from the VEVENT ✓

[flosch]

@rfc2822 said in DAVdroid can't handle events prior 1950:

Why? I have posted the data which is generated by DAVdroid and it seems to be correct:

Sorry I was unclear about that. I didn’t doubt the validity of the data itself DAVdroid generates. Deducing from your information, the Android calendar provider handles events due to how either DAVdroid or aCalDAV or whatever DAV-sync-app provide the information. Since events processed by aCalDAV are handled correctly whereas the very same events handled by DAVdroid aren’t, that’s why I doubt the Android calendar provider to be the problem here. It could be, though, that both the calendar provider and aCalDAV stick to some non-standard protocol to transfer information whereas DAVdroid sticks to standard protocols the calender provider unfortunately doesn’t understand. But still, the events get rendered correctly between aCalDAV and the Android calendar provider which, by logic, exclude both from being the problem. Am I wrong? [mebeingconfused]

I will provide the required debug information as soon as possible. Meanwhile: using Android 5.0.2, Calendar 4.40.17, DAVdroid 1.7.1-ose and aCalDAV 0.1.1.

[flosch]

Please find here the Debug info of DAVdroid. Just tried connecting to the CalDAV server via the internal LAN address instead of the internet which makes things even worse: now DAVdroid was unable to render the 1950 event as well. Randomly tried 1970 and it won’t show either. Only very recent events worked—very mysterious.

[rfc2822]

@flosch Then it seems quite clear to me: the server doesn’t send the event in the reply for the REPORT calendar-query of DAVdroid, which only requests event for 90 days into the past (+ forever in the future). The server has to calculate start and end date of recurring events and report them accordingly.

It also explains why the events show up in aCalDAV and Thunderbird: those probably don’t use REPORT calendar-query, but other methods like a worse-performing general PROPFIND instead (where the server doesn’t do any filtering).

To be sure please try to set the “Limit sync to X days in the past” setting in the DAVdroid account and leave the setting blank (default value: 90 days). Then sync again. If the event appears then, it’s clearly a server bug. By the way, which server do you use? This is a very important piece of information (as mentioned on https://forums.bitfire.at/topic/749/read-before-posting-what-s-required-to-diagnose-a-problem).

[flosch]

Colour me impressed! Leaving “Limit sync to X days in the past” blank really did the trick. To be honest, I doubted that solution because addressing the server via the internet (dyndns equivalent) instead of the home network, events past 1950 and not only the very recent showed up. And 1950 clearly is more than 90 days in the past! How come this is interpreted differently if either I address the server via the internet or via the internal LAN? The server is a Synology WebDAV, Version 2.3.3-0024, by the way. I wouldn’t expect much from filing a bug, however, since Synology is really slow, if caring at all.

As yet I was just testing DAVdroid. Hence the free version. Now that everything seems to work, I will move to the non-free version to support development. Thanks again for your help, well done.

Edit: is there a way to mark this solved?

[rfc2822]

@flosch I have set it to solved. Please report the bug to Synology nevertheless and provide the synology ticket ID for reference here, if possible, so we can mention it when someone encounters this problem in the future again.

[flosch]

I shall consider it. Two more questions off-topic:

• do the free version 1.7.1-ose and the non-free version 1.7.1 of DAVdroid differ? If not, I will stick to the ose version and donate instead.
• bying the app over amazon is EUR 3.19; how much do you (the DAVdroid developer(s)) get and what is Amazon’s share? Because the latter would be an additional reason for me to not buy the app but instead stick to the ose version and donate.

[devvv4ever]

@flosch There is no difference in features (only a donation dialogue on startup)! The shops always take 30%. So in any case we prefer donations Thanks in advance!

[flosch]

Thanks for the clarification. It’ll be a donation, then!