Clicking and dragging in either a positive (up/right) to zoom in and negative (down/left) to zoom out. It's not a difficult thing to do and I've seen it done before (not as part of a GUI but in scaling graphics manually).
The idea of using a the scroll-wheel is fine, but not all RISC OS users have access to them. Also, many RISC OS users are quite comfortable with drag-and-drop - don't know how well it'd go down on other OSes though.
I don't feel that dialog boxes should be over-used as part of the global workings of a GUI. They're fine within the realms of user applications but I wouldn't want them popping up all over the screen every time I wanted to zoom into several windows. Let's face it, most of them look butt-ugly and get really annoying (having to move them around, close them, etc.)