Alter User Agent: Remove 'okhttp'


  • As of the current builds available, DAVx will report to any WebDAV server with a user-agent containing okhttp followed by its version. This, while presumably done for some extra level of potential compatibility for the library interacting with websites, actually worsens compatibility for DAVx in some cases.

    Something I have found is that okhttp shows up in a popular filterlist for potentially malicious user agents. An issue was raised and decided to not remove the string for okhttp from said list, in light of more legitimate uses of the okhttp library such as DAVx in this case. What makes this issue more prevalent for many DAV servers, is that a popular nginx implementation, Bunkerized-Nginx, uses this filterlist in its default security config, meaning that any servers that wish not to apply specific security tuning settings or pull back the specific security feature outright won’t allow DAVx to function on those web servers.

    Since by default, its inclusion causes compatibility problems, I see no reason to have the section included in DAVx’s user agent. If the section’s inclusion in the user-agent is deliberate for known compatibility issues without it, a toggle in the settings to add in/revoke the okhttp section could also be another approach to fixing the problem.

  • developer

    @uocten Hi,

    I have included it purely for information. For instance, debugging HTTP problems (like the HTTP/2 problems which occured when HTTP/2 was still fresh) should be easier when you know which client library is used.

    See also https://datatracker.ietf.org/doc/html/rfc2616#section-14.43 and especially the example, which includes libwww:

      User-Agent: CERN-LineMode/2.15 libwww/2.17b3
    

    Has there actually been a problem with that? Would it be possible to whitelist DAVx5 + okhttp in this nginx blocklist?


  • @rfc2822 Hello,

    I can indeed confirm that testing with a webserver using Bunkerized-Nginx as part of a suitable WebDAV server, that DAVx will get its requests blocked due to its user agent matching to ‘okhttp’.

    Crawler-Detect filterlist

    As far as modding the filterlist itself, if the software using the filterlist allows passing a custom list location through, then it wouldn’t be hard to extract the line for ‘okhttp’ and remove it. The maintainer of the filterlist does not plan to change the list on its GitHub repo to remove ‘okhttp’.

    Bunkerized-Nginx

    For Bunkerized-Nginx specifically, the script that calls the filter list is static and calls the raw.githubusercontent page directly for sourcing it. Furthermore, no mechanism exists currently in the project to add exceptions to either of the two filterlists it pulls in for user agents. Making the primary option to fix the compatibility issue would be to disable user agent filtering entirely.

    For circumventing the mechanism, the most consistent option would be to overwrite the resulting file /etc/nginx/user-agents.list inside of the running Docker/Podman instance of Bunkerized-Nginx, after the container has started and has finished its startup process. This circumvention will need to be done every time the container starts up, and it needs to wait until the container’s internal startup scripts finish before overwriting the file. This isn’t really a great solution as it requires to poke into a container regularly from the outside whilst it is running, as well as detection for Bunkerized-Nginx’s startup process needs to be monitored.

    Another potential option would be to overwrite the script adding the filter list, though without the changes that need to be made being pushed to the github repo for Bunkerized-Nginx, they are likely to be overwritten any time the container updates, which may happen as much as every time an instance of the container is started. This is less favorable as it holds similar issues as the other circumvention as well as modifying the script should have more care put into it, as things may break when the source for the script updates.

Similar topics