Is this perhaps only happening with explorer windows?
Unfortunately no, it happens with any app
This may impact Groupy as the behaviour of borders
I don't think so — I also have Stardock Groupy 1.5 installed on the same system and setup, and it shows border correctly:

The latest build of Groupy has a tweak but it will only impact explorer windows as that was the only windows impacted and only if the tabs were not integrating with the titlebar which is what would normally happen on Windows 11.
Please note we would not recommend altering your OS to bring back the ribbon on Windows 11. This is not something we run or provide support on.
Also having both Groupy 2 and 1 installed will not work. It is very much a case of one or the other as they share configs.
Groupy 1.x also does not support Windows 11.