Furrygoat muses on the future of USB keychains. He mentions some strategies for running XP from USB via VM images. This could be done with various VM products including VM Player, which we have mentioned previously.
He has also championed the idea of 'portable CE'. This is running CE apps from a USB keychain via emulation.
There are a lot of resources floating around the web on how to get various apps (like Firefox) to run directly from flash. Many apps however don't lend them selves to this. More work will go on however as this idea becomes more popular. I find the prospect of carrying your favorite data and apps around with you very appealing. Another thing that would be nice would be for all your 'state' to be with you as well. This would be OS and app preferences, desktop settings etc. We are a little bit further away on this one.
Lastly a plug for a related idea that I've been hearing from some of my friends; In the wake of the recent hurricanes and other natural disasters, many people have found themselves not only without their homes and possessions, but without their important identity documents like birth certificates, social security cards, etc. The proposed solution is to scan these documents and store a copy via CD in a safe deposit box or with relatives. A variation is to keep them with you at all times on a USB keychain. I know a few people that do this, including photos of their family members for identification purposes. Of course security of this information would be a concern. To prevent identity theft I would strongly recommend encrypting this personal data somehow.