The latest version of PowerGUI is capable of installing to a USB key. Currently, to install PowerGUI to that key, you have to run a script that is installed into the PowerGUI directory. Dmitry outlined how to do this on his blog. This is pretty simple but definitely seems like something that should be integrated into the Script Editor. Because of this I created a Script Editor add-on that adds a command to call this script.
Script Editor Add On for Installing PowerGUI to a USB Key | Adam Driscoll’s Blog