(Discussion split from here.)
The most efficient solution would be to ask the server ala RFC 7809 whether it knows the time zones and not sent them in such case.
Besides, a time zone can have more than one name, which is currently in the X-LIC_LOCATION property. The Olson database is converted to .ics files using a “vzic” tool. I know of three vzic tools:
The first is not updated. The last one creates ics files that are slightly different from the middle one, but it utilizes the TZ-ALIAS-OF (RFC 7808) property to define a second name for a time zone. The precise differences are described at https://github.com/cyrusimap/cyrus-imapd/issues/3006#issuecomment-706978315. In short, the cyrus-vzic uses standard way to present the information, that is behind X-LIC_LOCATION.
@Дилян-Палаузов said in Creating/editing events no longer works for Fastmail:
The most efficient solution would be to ask the server ala RFC 7809 whether it knows the time zones and not sent them in such case.
Do you know how many servers support RFC 7809?
Also, I wonder whether this is applicable here. For instance, when a user enters a recurring event at 9:00 Europe/Vienna (Android), the Android VTIMEZONE should be sent, because it could differ from other definitions, for instance when the device is not 100% up to date and contains an old time zone definition (think about the case when some country abolishs DST).
In the given case, Android will calculate all instances of the recurring event with its own timezone definition. If you then define an exception, the recurrence ID will rely on the Android timezone definition, too.
If you replace the Android timezone definition by some other definition, there are cases which will produce unwanted results, for instance that a recurrence ID that matches on the Android device won’t match on the server.
I don’t think RFC 7809 is useful if the client’s (here: Android/JVM) time zone database can’t be synchronized with the distribution service.
I have no idea how distributed are RFC 7809 server implementitons, but I can provide test account with the service.
This is very much applicable here, because the CUA would not have sent VTIMEZONE and the server and others would not have had headache, because the sent VTIMEZONE has two X- properties.
There are many, many different places that distribute timezone information, having all as source the Olson database: the operating system, pytz, the CalDAV server, ICU. The problem is, that whichever distribution is taken, at some moment just it might not be updated and every other thing might get updated. The introduction of RFC 7808 says it:
Calendaring and scheduling systems, such as those that use iCalendar [RFC5545], as well as operating systems, critically rely on time zone data to determine the correct local time. As such, they need to be kept up to date with changes to time zone data. To date, there has been no fast and easy way to do that. Time zone data is often supplied in the form of a set of data files that have to be “compiled” into a suitable database format for use by the client application or operating system. In the case of operating systems, often those changes only get propagated to client machines when there is an operating system update, which can be infrequent, resulting in inaccurate time zone data being present for significant amounts of time. In some cases, old versions of operating systems stop being supported, but are still in use and thus require users to manually “patch” their system to keep up to date with time zone changes.
If the time zones definitions of the server are used and are outdated, you are in trouble. If instead you use the timezone definitions of the operating system, these are outdated, and the timezone definitions of the server are ignored, you are in no better position. You use one source and it can be updated or outdated.
In case of Cyrus Imap/Fastmail it strips the VTIMEZONEs, that are sent together with VEVENT from the VCALENDAR, if it knows the TZID: https://github.com/cyrusimap/cyrus-imapd/blob/master/imap/http_caldav.c#L8012 . So DAVx⁵ sends the time zone definitions and the server strips these and does not return it on GET. I am proposing just not to send the timezone data.
@Дилян-Палаузов said in Creating/editing events no longer works for Fastmail:
In case of Cyrus Imap/Fastmail it strips the VTIMEZONEs, that are sent together with VEVENT from the VCALENDAR, if it knows the TZID: https://github.com/cyrusimap/cyrus-imapd/blob/master/imap/http_caldav.c#L8012 . So DAVx⁵ sends the time zone definitions and the server strips these and does not return it on GET. I am proposing just not to send the timezone data.
So for instance if the EU drops DST next year, and Android uses the old timezone and sends an event at 1.5.2021 10:00 Europe/Vienna (old) = 8:00 UTC, this event will be shown as 10:00 Europe/Vienna (new) = 9:00 UTC in other clients? And users don’t know when the “shift” in the UTC time applies and when it doesn’t?
What if it’s a recurring event and there is a recurrence ID for a specific date on 8:00 UTC?
At the end it is “DTSTART;TZID=Europe/Vienna:20210501T09:00:00”. I do not propose a world-saving plan, when the OS is not informed, that DST was dropped. Using an OS that is not aware of DST in the TZ of the user will have some implications for the user and the user has to accept the implications, or update. Such TZ changes do happen in very short-term, than ICU makes a new release, read about Marocco at https://ftp.iana.org/tz/tzdb-2020b/NEWS (e.g for 2020a). I say, that not sending around the time zone definitions saves bandwidth and the server can strip anyway the received timezone definitions. If you want, that the server does not strip these, use cryptic names for TZID. If you want to get the timezone definitions on GET, send CalDAV-Timezones: T (or rather the server definitions of the definitions, not the one sent). The server can calculate the freebusy state of a user, based on TZID in DTSTART/DTEND, its understanding of the timezone definitions and its knowledge about leap seconds.
The RECURRENCE-ID must be specified with local time if and only if DTSTART was specified with local time. My reading is that either both use floating time (without TZID), or both use the same TZID, so the recurrence matches do not depend on the actual timezone/dst definitions.
@Дилян-Палаузов I still think that RFC 7809 is only useful when you’re in full control of the used timezones (for instance, when you use the Java Time API and can choose another timezone database then the system DB, which is not the case for DAVx5, because the Android system will calculate recurrences, time zone shifts etc), but maybe I have missed something. I’ll have a look at it when I have more time.
Also, I’m quite sure that most servers (especially those with most of our users) don’t implement RFC 7809 yet, so it may be an additional feature, but doesn’t free us from generating correct VTIMEZONEs.
Irrespective of the server support it might be a good idea to add a per account/global option “Do not send timezone descriptions to the server, when uploading data”. This can be used to evaluate whether this is generally a good idea and if so, turn that option by default on in the future. I personally have no experience with using timezone data, that does not match reality. Most persons will likely never have this problem. Once timezone data is cut from VCALENDAR all clients are affected, even those who do not support tzid by reference. I want to offer public calendars, where many users subscribe to event streams, the events has dtstart with tzid, and the CUAs do not get the timezone definitions from the server (unless they ask explicitly for it). I do not expect any troubles. As far as I understood it, DAVx5 uploaded timezone data, that was not updated for two years.
I personally have no experience with using timezone data, that does not match reality. Most persons will likely never have this problem.
The problem are not most users. I don’t want broken and shifted events even for 1% (not only because it would result in many 1* ratings, but because users should get reliable synchronization; within the means of what is possible).
Playing around with VTIMEZONEs requires to be absolutely sure what you’re doing and also many compatibility tests with dozens of servers (you see what happens if those tests are not done carefully – even adding two X- properties in VTIMEZONEs causes broken sync with many servers). Implementing RFC 7808/7809 means much work to change something that is already working. I don’t say that RFC 7808/7809 is bad, I just need to understand it and its consequences.
By the way, where does RFC 7809 say that servers may strip VTIMEZONE data when TZID by reference is not active? I can’t find that. As I understands it, it defines a new mode where client and server can agree to use a shared, synchronized VTIMEZONE database (Time Zone Data Distribution Service, RFC 7808).
In RFC 7808 2. Architectural Overview:
(e) Clients: Applications, operating systems, etc., that make use of
time zone data and retrieve that from either root or secondary
providers.
Wouldn’t that mean that DAVx5, the Android calendar provider and Android calendar apps (= altogether the “client”) would have to use the synchronized database? Unfortunately, this is not only within the scope of DAVx5 alone.
Let’s say one CUA (calendar user agent) uploads an iCalendar object and a second CUA downloads the same iCalendar object. The second CUA gets no VTIMEZONEs in the object, but it cannot know, whether the VTIMEZONEs are missing, because they were not uploaded by the first CUA (that possibly understands RFC 7809), or because the first CUA uploaded the VTIMEZONEs and the server cut them. This is what happens currently, when DAVx5 is the first or the second CUA: the other CUA (possibly DAVx5) does not see VTIMEZONEs for whatever reason. This has not led to negative feedback towards the DAVx5 product.
DAVx5 used the timezone definitions from ical4j, which were switched recently from Olson 2018g to Olson 2020b. DAVx5 has uploaded for a year possibly outdated timezone 2018g definitions and this has not lead to noticable impact on the users.
To summarize: currently nobody has recognized a problem, when the servers deliver iCalendar objects, that do not contain VTIMEZONE definitions. Neither missing nor outdated timezone definitions lead to negative ratings for DAVx5. The discussion here is about whether the CUA should refrain from bundling the time zone definitions on upload. The discussion is not about exchanging pure timezone information.
@Дилян-Палаузов said in Implement RFC 7809 (Time Zones by Reference):
uploaded the VTIMEZONEs and the server cut them. This is what happens currently, when DAVx5 is the first or the second CUA: the other CUA (possibly DAVx5) does not see VTIMEZONEs for whatever reason. This has not led to negative feedback towards the DAVx5 product.
That may be true for the Cyrus server.
But who says that those servers: https://www.davx5.com/tested-with/ (and many more) will accept VEVENTs without VTIMEZONE although RFC 5545 clearly requires that (An individual “VTIMEZONE” calendar component MUST be specified for each unique “TZID” parameter value specified in the iCalendar object.)?
DAVx5 used the timezone definitions from ical4j, which were switched recently from Olson 2018g to Olson 2020b. DAVx5 has uploaded for a year possibly outdated timezone 2018g definitions and this has not lead to noticable impact on the users.
Timezone-related problems occur from time to time.
And sending the ical4j timezones is not exactly accurate. As I see it, DAVx5 should send the timezone definitions of the system instead, because those are the ones which are used for calculations on the Android system (but I haven’t done that yet).
Neither missing nor outdated timezone definitions lead to negative ratings for DAVx5.
Again, how do you measure that? Did you test all servers and send RFC 5545-violating VEVENTs without VTIMEZONE and it worked without problems? I’m very interested in such test results.
In the case that no tested server makes problems, it’s worth thinking of, but it’s still a violation of RFC 5545. And RFC 7809 is only applicable when both client and server support it and agree on it. Shouldn’t DAVx5 be RFC 5545-compliant?
As not-uploading VTIMEZONE has both pros and contra, and uploading VTIMEZONE has also pros and contra, the most reasonable step would be to offer a per account/global option to the user “Shall VTIMEZONE data be uploaded?” After a while experimenting with this option, more information and experience can be collected.
It would be a further improvement, if for servers returning DAV:calendar-no-timezone in OPTIONS on a calendar-home-set, that option is automatically enabled.
RFC 5545 was written in 2009. Today operating systems have built-in knowledge about about Olson database. When the timezone is both not in Olson DB but widespread, e.g. Europe/FLE, then it is still in the VCALENDAR.
As long as Morocco continues to change its timezone rules on short-term, users will experience “unexpected” shifts.
If the proposed change, that is a matter of fact in effect, has led to negative feedback for DAVx5, I guess this would have been already taken into account.
Compare the discussion of “May the CUA not send the VTIMEZONE data, when the server would strip that information anyway?”, to the discussion “What do you think about CUAs, which insists form the users to provide a user name and password, in order to complete the CalDAV setup, but never send the credentials to the server, unless the server returns 401 Unauthorized, and the server may never return 401 Unauthorized, if it offers both authenticated and not authenticated (anonymous) CalDAV access?”
Here one current example: I schedule in August 2020 a meeting, targeted for Fuji, for 2020-11-11 and send an .ics invitation, bundling tz definitions. Afterwards the Fuji experts change DST rules, so that the switch does not happen on 2020-11-08 but on 2020-12-20. The UTC offset calculations, included in the .ics invitation, are therefore for that meeting wrong. Nobody thinks on regenerating and resending the .ics file with new tz data. If the OS gets updates, then that event might be presented correctly, if the .ics interpreter ignores the bundled TZ definitions, if the .ics interpreter sees no bundled timezone definitions, or if the .ics interpreter shows to the user, that the bundled TZ definitions are outdated, and there are two possible times for the start.
There is no ultimate right way on technical level to prevent troubles here, not bundling the time zone definitions in the .ics file saves band width.
See also https://techcommunity.microsoft.com/t5/daylight-saving-time-time-zone/fiji-dst-end-date-correction-update-now-available/ba-p/1669658 and https://techcommunity.microsoft.com/t5/daylight-saving-time-time-zone/interim-guidance-on-2020-time-zone-updates-for-fiji/ba-p/1766806 .
