I guess if you do the installation on the USB drive
(with nothing to do with the main drive) at boot time
you can hit one of the "escape keys" to take you to
the boot sequence chooser. I've got an IBM ThinkPad T42p
and I hit F12 to access it. Check your manufacturer to
discover that key and once you turn on your computer
hit that key and it will take you to that menu, from which
you can select where to boot, in this case...the USB.
I have never done it, but it should be simple.

I've got two disks; one with Ubuntu and the other
with some other crap I need for work. So, one is
on the main hard drive caddy, the other on the
removable caddy. At boot time, I hit F12 and can
choose the removable. If I don't hit F12, it will boot
directly from the main hard drive. No GRUB.