Thanks for the information. I’ll close this issue now because the problem seems to be resolved.
ampersands in caldav cause parsing error
-
Hello,
Thanks for the report. As far as I know, ampersands have to be encoded as entities in XML, don’t they?
If you agree, can you please report this to SOGo and provide a link for reference here?
-
As far as I know, ampersands have to be encoded as entities in XML, don’t they?
Yes, and they are (open the attachment in a text editor to see this, it seems the forum opens them inline, thus decoding entities)
To avoid confusion, i include them inline:
Working
<?xml version="1.0" encoding="utf-8"?> <D:multistatus xmlns:D="DAV:" xmlns:a="urn:ietf:params:xml:ns:caldav"> <D:response> <D:href>/SOGo/dav/foo@example.net/Calendar/personal/</D:href> <D:propstat> <D:status>HTTP/1.1 200 OK</D:status> <D:prop></D:prop> </D:propstat> <D:propstat> <D:status>HTTP/1.1 404 Not Found</D:status> <D:prop> <a:calendar-data/> </D:prop> </D:propstat> </D:response> <D:response> <D:href>/SOGo/dav/foo@example.net/Calendar/personal/19A7-5CAA4180-15-5207B580.ics</D:href> <D:propstat> <D:status>HTTP/1.1 200 OK</D:status> <D:prop> <a:calendar-data>BEGIN:VCALENDAR PRODID:-//Inverse inc./SOGo 4.0.7//EN VERSION:2.0 BEGIN:VTIMEZONE TZID:Europe/Paris X-LIC-LOCATION:Europe/Paris BEGIN:DAYLIGHT TZOFFSETFROM:+0100 TZOFFSETTO:+0200 TZNAME:CEST DTSTART:19700329T020000 RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:+0200 TZOFFSETTO:+0100 TZNAME:CET DTSTART:19701025T030000 RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:19A7-5CAA4180-15-5207B580 SUMMARY:foo CLASS:PUBLIC X-SOGO-SEND-APPOINTMENT-NOTIFICATIONS:NO TRANSP:OPAQUE DTSTART;TZID=Europe/Paris:20190408T100000 DTEND;TZID=Europe/Paris:20190408T104500 CREATED:20190407T182922Z DTSTAMP:20190407T182922Z LAST-MODIFIED:20190407T182922Z END:VEVENT END:VCALENDAR</a:calendar-data> </D:prop> </D:propstat> </D:response> </D:multistatus>
Not working
<?xml version="1.0" encoding="utf-8"?> <D:multistatus xmlns:D="DAV:" xmlns:a="urn:ietf:params:xml:ns:caldav"> <D:response> <D:href>/SOGo/dav/c&m@example.net/Calendar/personal/</D:href> <D:propstat> <D:status>HTTP/1.1 200 OK</D:status> <D:prop></D:prop> </D:propstat> <D:propstat> <D:status>HTTP/1.1 404 Not Found</D:status> <D:prop> <a:calendar-data/> </D:prop> </D:propstat> </D:response> <D:response> <D:href>/SOGo/dav/c&m@example.net/Calendar/personal/3E5C-5CAB8E00-35-469F7780.ics</D:href> <D:propstat> <D:status>HTTP/1.1 200 OK</D:status> <D:prop> <a:calendar-data>BEGIN:VCALENDAR PRODID:-//Inverse inc./SOGo 4.0.7//EN VERSION:2.0 BEGIN:VTIMEZONE TZID:Europe/Paris X-LIC-LOCATION:Europe/Paris BEGIN:DAYLIGHT TZOFFSETFROM:+0100 TZOFFSETTO:+0200 TZNAME:CEST DTSTART:19700329T020000 RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:+0200 TZOFFSETTO:+0100 TZNAME:CET DTSTART:19701025T030000 RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:3E5C-5CAB8E00-35-469F7780 SUMMARY:foo CLASS:PUBLIC X-SOGO-SEND-APPOINTMENT-NOTIFICATIONS:NO TRANSP:OPAQUE DTSTART;TZID=Europe/Paris:20190408T100000 DTEND;TZID=Europe/Paris:20190408T104500 CREATED:20190407T182922Z DTSTAMP:20190407T182922Z LAST-MODIFIED:20190407T182922Z END:VEVENT END:VCALENDAR</a:calendar-data> </D:prop> </D:propstat> </D:response> </D:multistatus>
-
I see. Can you please provide the full DAVx⁵ exception info (which includes details about the exception, a stack trace etc)? This sounds really strange, because DAVx⁵ uses the Android XML parser and doesn’t parse by itself… Steps to reproduce would also be helpful.
-
Here is the log:
EXCEPTION: at.bitfire.dav4jvm.exception.DavException: Couldn't parse multistatus XML element at at.bitfire.dav4jvm.DavResource.processMultiStatus(DavResource.kt:494) at at.bitfire.dav4jvm.DavResource.processMultiStatus(DavResource.kt:435) at at.bitfire.dav4jvm.DavCalendar.calendarQuery(DavCalendar.kt:99) at at.bitfire.davdroid.syncadapter.CalendarSyncManager$listAllRemote$2.invoke(CalendarSyncManager.kt:108) at at.bitfire.davdroid.syncadapter.CalendarSyncManager$listAllRemote$2.invoke(CalendarSyncManager.kt:42) at at.bitfire.davdroid.syncadapter.SyncManager.useRemote(SyncManager.kt:809) at at.bitfire.davdroid.syncadapter.SyncManager.useRemoteCollection(SyncManager.kt:830) at at.bitfire.davdroid.syncadapter.CalendarSyncManager.listAllRemote(CalendarSyncManager.kt:106) at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1$1.invoke(SyncManager.kt:142) at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1$1.invoke(SyncManager.kt:53) at at.bitfire.davdroid.syncadapter.SyncManager.syncRemote(SyncManager.kt:455) at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1.invoke(SyncManager.kt:141) at at.bitfire.davdroid.syncadapter.SyncManager$performSync$1.invoke(SyncManager.kt:53) at at.bitfire.davdroid.syncadapter.SyncManager.unwrapExceptions(SyncManager.kt:835) at at.bitfire.davdroid.syncadapter.SyncManager.performSync(SyncManager.kt:112) at at.bitfire.davdroid.syncadapter.CalendarsSyncAdapterService$CalendarsSyncAdapter.sync(CalendarsSyncAdapterService.kt:55) at at.bitfire.davdroid.syncadapter.SyncAdapterService$SyncAdapter.onPerformSync(SyncAdapterService.kt:73) at android.content.AbstractThreadedSyncAdapter$SyncThread.run(AbstractThreadedSyncAdapter.java:321) Caused by: org.xmlpull.v1.XmlPullParserException: unterminated entity ref (position:TEXT @2:104 in okhttp3.ResponseBody$BomAwareReader@be74a6) at org.kxml2.io.KXmlParser.readEntity(KXmlParser.java:1220) at org.kxml2.io.KXmlParser.readValue(KXmlParser.java:1402) at org.kxml2.io.KXmlParser.next(KXmlParser.java:393) at org.kxml2.io.KXmlParser.next(KXmlParser.java:313) at org.kxml2.io.KXmlParser.nextText(KXmlParser.java:2077) at at.bitfire.dav4jvm.Response$Companion.parse(Response.kt:113) at at.bitfire.dav4jvm.DavResource$processMultiStatus$2.invoke(DavResource.kt:465) at at.bitfire.dav4jvm.DavResource.processMultiStatus(DavResource.kt:484) ... 17 more
-
as an addition, the error
Caused by: org.xmlpull.v1.XmlPullParserException: unterminated entity ref (position:TEXT @2:104 in okhttp3.ResponseBody$BomAwareReader@be74a6)
refers to the first letter in the xml entity:
<?xml version="1.0" encoding="utf-8"?> <D:multistatus xmlns:D="DAV:" xmlns:a="urn:ietf:params:xml:ns:caldav"> <D:response> <D:href>/SOGo/dav/c&m@example.net/Calendar/personal/</D:href> ↑ This is 2:104 in raw output
@rfc2822 If required, I can give you a temporary access to this calendar
-
@CircleCode At the moment, we don’t have any left time. If you can keep the test account online some time, please create one and send it over forum chat. Thanks
-
for the reference, this may be related to https://sogo.nu/bugs/view.php?id=2479
As an addition, I am wondering if entities are expected at all in href (at xml level, they should be, of course, but at the caldav parser, I am wondering).
For example, when I share this calendar with a user not having an ampersand in its name, it becomes as following (note the replacements :&
→_AM_
,@
→_A_
,.
→_D_
) :<?xml version="1.0" encoding="utf-8"?> <D:multistatus xmlns:D="DAV:" xmlns:a="urn:ietf:params:xml:ns:caldav"> <D:response> <D:href>/SOGo/dav/user@example.net/Calendar/c_AM_m_A_example_D_net_personal</D:href> <D:propstat> <D:status>HTTP/1.1 200 OK</D:status> <D:prop></D:prop> </D:propstat> <D:propstat> <D:status>HTTP/1.1 404 Not Found</D:status> <D:prop> <a:calendar-data/> </D:prop> </D:propstat> </D:response> <D:response> <D:href>/SOGo/dav/user@example.net/Calendar/c_AM_m_A_example_D_net_personal/3E5C-5CAB8E00-35-469F7780.ics</D:href> <D:propstat> <D:status>HTTP/1.1 200 OK</D:status> <D:prop> <a:calendar-data>BEGIN:VCALENDAR PRODID:-//Inverse inc./SOGo 4.0.7//EN VERSION:2.0 BEGIN:VTIMEZONE TZID:Europe/Paris X-LIC-LOCATION:Europe/Paris BEGIN:DAYLIGHT TZOFFSETFROM:+0100 TZOFFSETTO:+0200 TZNAME:CEST DTSTART:19700329T020000 RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:+0200 TZOFFSETTO:+0100 TZNAME:CET DTSTART:19701025T030000 RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:3E5C-5CAB8E00-35-469F7780 SUMMARY:Mat CLASS:PUBLIC X-SOGO-SEND-APPOINTMENT-NOTIFICATIONS:NO TRANSP:OPAQUE DTSTART;TZID=Europe/Paris:20190408T100000 DTEND;TZID=Europe/Paris:20190408T104500 CREATED:20190407T182922Z DTSTAMP:20190407T182922Z LAST-MODIFIED:20190407T182922Z END:VEVENT END:VCALENDAR</a:calendar-data> </D:prop> </D:propstat> </D:response> </D:multistatus>
-
@rfc2822 I won’t be able to keep the test account online for time.
For reference:
I am using SoGo 4.0.7 on Gandi hosting.
I did nothing else than creating a mail with address c&m@example.net, and it should be sufficient to reproduce this issue. -
Can you please send a test account nevertheless? I’ll have a look as soon as I find some time.
-
I did not find a way to send privte message, and I’d like to avoid posting credentials in the forum
-
may be related to https://sogo.nu/bugs/view.php?id=2479, but not sure
-
Thanks for the test account. When DAVx⁵ queries the resources, the server sends:
<?xml version="1.0" encoding="utf-8"?> <D:multistatus xmlns:a="urn:ietf:params:xml:ns:carddav" xmlns:D="DAV:"><D:response><D:href>/SOGo/dav/c%26m@example</D:href><D:propstat><D:status>HTTP/1.1 200 OK</D:status><D:prop><a:addressbook-home-set><D:href xmlns:D="DAV:">/SOGo/dav/c&m@example/Contacts/</D:href></a:addressbook-home-set><D:group-membership/></D:prop></D:propstat></D:response></D:multistatus>
which causes this DAVx⁵ error:
Couldn't parse urn:ietf:params:xml:ns:carddavaddressbook-home-set EXCEPTION org.xmlpull.v1.XmlPullParserException: unterminated entity ref (position:TEXT @2:247 in okhttp3.ResponseBody$BomAwareReader@abfda3c)
As you can see, the & is not escaped in the href element, which is invalid XML and can’t be parsed. So, this seems to be a server problem and it seems to be related to the bug report you have posted.
If there are any specific indications that this could be a DAVx⁵ problem, please let us know.