DAVx⁵ user-agent filtered by opnsense/nginx

[molusk]

Hi,
I ran into an issue when I migrated my Nextcloud reverse proxy from Apache2 (on my NAS) to ngnix on an opnsense firewall : my phone, and only it, could not synchronize contacts and calendar anymore. No issue with Nextcloud app, nor with contacts and calendar sync with Thunderbird or Windows 10 Mail app.

The error I got is a simple 403 response with a message telling I was banned and my mobile phone IP address was in the ban list on opnsense (in nginx module) but Nextcloud app still worked, it wasn’t banned.

So I looked at the user-agent DAVx⁵ was using to compare with le bot blocklist I read mentions of in one of the many sites, forum, faq… I read.

DAVx⁵ user-agent is :

DAVx5/3.3.7-ose (2020/11/30; dav4jvm; okhttp/4.9.0) Android/10\r\n

The code responsible of the issue in opnsense is here :
https://github.com/opnsense/plugins/blob/b2d5d685e32968604feea055b8d6285211d4d0e1/www/nginx/src/opnsense/service/templates/OPNsense/Nginx/http.conf#L220

{% if server.disable_bot_protection is not defined or server.disable_bot_protection != '1' %}
    # block based on User Agents - stuff I have found over the years in my server log
    if ($http_user_agent ~* Python-urllib|Nmap|python-requests|libwww-perl|MJ12bot|Jorgee|fasthttp|libwww|Telesphoreo|A6-Indexer|ltx71|okhttp|ZmEu|sqlmap|LMAO/2.0|ltx71|zgrab|Ronin/2.0|Hakai/2.0) {
      return 418;
    }
    {# MSIE 7 cannot be blocked - used for compatibility mode - https://blogs.msdn.microsoft.com/ieinternals/2013/09/21/internet-explorer-11s-many-user-agent-strings/ #}
    if ($http_user_agent ~ "Indy\sLibrary|Morfeus Fucking Scanner|MSIE [0-6]\.\d+")
    {
      return 418;
    }
    if ($http_user_agent ~ ^Mozilla/[\d\.]+$)
    {
      return 418;
    }
{% endif %}

As you can see, okhttp in the user-agent string triggers the bot protection option in opnsense nginx implemetation.

The workaround is really simple : disable bot protection for my nextcloud instance and it is okay until I open the service from outside.
I think I can develop a quick and dirty fix in opnsense configuration but not sure the community would accept the patch.

I will also open an issue on opnsense side. If you can’t do anything about the user-agent sent by DAVx⁵, at least maybe this post will allow others to understand their issue. I did not find a lot of usefull information anywhere, maybe my keywords were too generic or my issue is really specific.

If you want more details or if I am not clear feel free to ask, I can reproduce the issue really quickly until I make my dirty fix to be able to access my Nextcloud instance from outside my network.