AMS: Yes, tuning for XScale may have given a tiny advantage, possibly more than using XScale-specific instructions. The reason he was nominated for own goal is that he will have known full well that a) using XScale instructions would have made a trivial and unnoticable difference, b) that this would stop it working on other machines, c) somebody got it to work anyway, without his help. Seems completely reasonable to me.
TonyStill: The more features of the compiler you use, the more difficult it is to discover if the bug is in your code, or in the compiler's. Which is why most software is developed with -O0 and only built with more optimisations later. Using certain optimisations and compiler features can also make it impossible to debug the software.