Zimbra shared calendar: Some events are not synchronized

[rfc2822]

Which workaround?

Until now, I don’t know where the problem lies and what exactly it is, because I don’t have enough information and I have never seen it in real, nor in logs.

If you can provide a test account and reliable steps to reproduce the problem, I can have a look.

[McNetic]

I don’t know how the workaround works, I just concluded that there has to be a working solution as other sync tools can synchronize where DAVx5 fails. I will try to see if I can provide more input to help you diagnose the problem, but it may take some time…

[rfc2822]

@McNetic Thanks for your understanding! Please post new information here.

[McNetic]

Ok, I think found the cause of the problems: Appearently, if I connect shared calendars into a Zimbra account, Zimbra announces the URLs of the actual events differently depending on wether the Calendar URL contains Spaces (%20) or not:

If my Account URL is https://example.com/dav/user1@example.com/, and I have connected two shared calendars from user2, named “Test” and “Test 2”, then the events from the Calendar “Test” are announced for example as https://example.com/dav/user1@example.com/Test/26192814-cbe4-4ace-8350-e2b9f29c6e3c.ics, but the events from calendar “Test 2” are announced as https://example.com/dav/user2@example.com/Test 2/be722de3-ab97-4368-8cf3-45478c6ec523.ics (if you don’t spot it immediately: the URL with the space has the user the calendar is shared from in it, while the other one has my own account in it).

In Response.kt lines 180 and following, the event URLs are compared to the calendar base url, which always contains the account user, so these events are rejected as HrefRelation.OTHER.

This also explains the synchonization works for creating events on android and syncing them back to the server, only to have them then disappear locally.

Unfortunately, I don’t know much of the DAV standards, so I don’t know why the URLs are compared to the base at all, and wether the behaviour of zimbra is standards compliant or not. Obviously, my previous caldav synchronisation software ignored this differences and I never had problems with that behaviour.

If you are interested and would explain your intention on how to handle the situation, I might be able to provide a patch.

Thank you very much.

[rfc2822]

Thanks for your investigation.

@McNetic said in Zimbra shared calendar: Some events are not synchronized:

In Response.kt lines 180 and following, the event URLs are compared to the calendar base url, which always contains the account user, so these events are rejected as HrefRelation.OTHER.

Well at least they’re classified as relation: other. dav4jvm doesn’t reject anything

They’re just not useful, because PROPFIND (and REPORT calendar-query, but let’s simplify things and say we’re just using PROPFIND, which would have to be able to enumerate all resources, too) work on collections (= directories) and request members of (= files in) this collection:

https://tools.ietf.org/html/rfc4918#section-9.1

The PROPFIND method retrieves properties defined on the resource identified by the Request-URI, if the resource does not have any internal members, or on the resource identified by the Request-URI and potentially its member resources, if the resource is a collection that has internal member URLs.

—

Obviously, my previous caldav synchronisation software ignored this differences and I never had problems with that behaviour. If you are interested and would explain your intention on how to handle the situation, I might be able to provide a patch.

I wonder how they can do that, because I think it would break many things. I have seen servers which return nearly everything on PROPFIND requests, including parent collections, completely unrelated resources, the root collection etc. I’d like to stick with the standards, and as far as I know and understand it, returning resources which are not the requested resource or members of this resource doesn’t make any sense and parsing these responses can cause various errors. In DAVx⁵, they’re just used for resource detection (then resources with HrefRelation.OTHER are taken into consideration), [edit:] because properties like current-user-principal can be used from any source and it improves the results.

If you have other conclusive interpretations of WebDAV/CalDAV (which would include how such non-member resources could be handled), please let me know. These things touch the deep core of DAVx⁵ (and my understanding of WebDAV/CalDAV), so I’ll be very cautious with changing those things. However, if there’s some mistake, it will of course have to be fixed.

[McNetic]

@rfc2822 said in Zimbra shared calendar: Some events are not synchronized:

Well at least they’re classified as relation: other. dav4jvm doesn’t reject anything

Sorry, you’re right; the ‘rejection’ takes place later in SyncManager.kt, lines 458 ff.:

 // ignore non-members
            if (relation != Response.HrefRelation.MEMBER)
                return@listRemote

For a quick test, I just reduced the classification code to just consider location scheme, host and port for HrefRelation.MEMBER, and now, DAVx5 appears to behave like the other solution I used.

They’re just not useful, because PROPFIND (and REPORT calendar-query, but let’s simplify things and say we’re just using PROPFIND, which would have to be able to enumerate all resources, too) work on collections (= directories) and request members of (= files in) this collection:

https://tools.ietf.org/html/rfc4918#section-9.1

The PROPFIND method retrieves properties defined on the resource identified by the Request-URI, if the resource does not have any internal members, or on the resource identified by the Request-URI and potentially its member resources, if the resource is a collection that has internal member URLs.

I do not understand what that section of the rfc should tell me; but I as I said, I don’t know much about the DAV standards at all. But I also do not understand how classifying the resources based on the url als MEMBER or OTHER is related to working on collections/directories/files. My impression is, that DAVx5 requests the events in the specified time range from the server, and it responds with a list of events (and as far as I can see no other resources).

I wonder how they can do that, because I think it would break many things. I have seen servers which return nearly everything on PROPFIND requests, including parent collections, completely unrelated resources, the root collection etc. I’d like to stick with the standards, and as far as I know and understand it, returning resources which are not the requested resource or members of this resource doesn’t make any sense and parsing these responses can cause various errors. In DAVx⁵, they’re just used for resource detection (then resources with HrefRelation.OTHER is taken into consideration).

If you have other conclusive interpretations of WebDAV/CalDAV (which would include how such non-member resources could be handled), please let me know. These things touch the deep core of DAVx⁵ (and my understanding of WebDAV/CalDAV), so I’ll be very cautious with changing those things. However, if there’s some mistake, it will of course have to be fixed.

Unfortunately, I don’t see how I can provide other insights here. I already fail to understand where the difference betwenn MEMBER and OTHER comes from in the specs…

[McNetic]

I think I found the relevant section that disallows URLs not matching the request URL: https://tools.ietf.org/html/rfc4918#section-8.3

I now found that nearly this exact problem was already discussed three years ago here: https://forums.bitfire.at/topic/1270/allow-direct-dav-urls-without-auto-detection/, with bugs reported to zimbra which got only partially fixed. I reported the remaining problem here: https://bugzilla.zimbra.com/show_bug.cgi?id=109237.

I’ll still go with my patched version of DAVx5 for now, as all my shared calendar names include spaces and the feature is critical to me, and fixing the zimbra side is much more complicated (I don’t want to run a manually patched version of a public service).

[rfc2822]

@McNetic You can also see it like that: you can connect to a WebDAV share with a graphical client, let’s say Nautilus (or Explorer). When you navigate to /dir1, the client sends PROPFIND /dir1 and shows the results. What should the client do when the server does not only return /dir1/file1 and /dir1/file2, but /somewhere/else? Should it be shown as file in dir1 (as requested), or be ignored (because it really doesn’t belong to dir1, which has been requested)?

In dav4jvm, I chose it to be registered, but as “related resource”, because it was not requested, but it may contain useful information. In DAVx⁵, such related resources are only used for auto-detection purposes, but for synchronization, entries from other directories are ignored because they do not fit in the directory scheme.

Maybe you can discuss that on Zimbra side, too.

[McNetic]

Thank you for your reply. I think the zimbra developers agree on the issue, as there is code in place that should provide the correct hrefs, but fails to do so for this special case.

[rfc2822]

@McNetic Ok. Hopefully this can be fixed there. But if you find something specific in DAVx⁵ that can be improved safely, just let us know.