Go to pendrivelinux.com for tutorials on how to have a persistent usb key.
You can also install it to the usb key, but this is trickier because you
need to write grub to the key and not to the hard drive. I have done both
methods successfully using several distributions, so know they will work.
The second method will give you a full installation and not just a live
disk with persistence.

You can also use a usb key and downloaded ISO to make a bootable live
installer using Unetbootin. This is my preferred method for installing and
seldom resort to a burned disk.