I presume you know what a Zip file/folder is and know how you would use that zipped file/folder.
If you don't, I'll try explain. It is a file/folder that has been compressed to make it smaller. Downloading is quicker using a smaller, zipped file/folder.
You need to unzip/extract the file/folder to reveal it's contents. These files are the ones you need to use. Remove the zipped ones as they are not required any more.
This can be done any where you like, even on the Flash (Pen) drive that you plan to use.
The whole idea of the 'Portable' applications is that you don't actually put them on the other PC. If this was the case, why bother with the portable concept.
Give it a go yourself, no harm done by trying. You may even be able to post back and tell us all how you got on!