I have contacts stored in ownCloud with PHOTO and other fields like X-GENDER and BDAY. After making changes to a contact on the phone those fields have disappeared in ownCloud. DAVDroid should keep fields it doesn’t know.
The biggest reasons to keep your Google Calendar around would be that most other people use it and it also does an amazing job scrapping my GMail (kind of shudder worthy if you think about it) for events to automatically add to the calendar. Think of a friend sharing an upcoming event of theirs or a flight you got an email confirmation of. I THINK I can work around this by adding Google’s Calendar to NextCloud as a subscription just to keep an eye on new events popping up.
Contacts is a bit more of a hassle but I think I can live with it. Google, again, will pull information from anything it can to fill in contact information, such as from GMail or your phones Contact app. Most times, where do your modify your contacts? On your phone. In the end I don’t think it is as important to have my contacts in NextCloud as it is calendars, since it is not like I am emailing or texting from NextCloud, their app already sends backups to NextCloud and I can live with this.