And just after I posted that I thought of trying something I hadn't tried before. Here is where the problem is.
If you add a new category to the dock by right clicking and then going to Add> Category and choose an icon from the dialog for it it applies fine and appears as it does in the dialog window.
If you drag and drop something onto the dock and then go to edit to try and use one of the included icons then it does this.