So this issue has been requested for a few years, but still hasn't been addressed. Folder portals always show their labels on mouseover, even when in the settings labels are set to "Never".
Can you please find a solution to having the folder portal label keep showing? Someone who doesn't want their fences to show labels will of course not want folder portals to do it either. But so far we've been stuck with it.