I have tried using the lastest version of the Portable Virtualbox Starter 5.0 with the new beta of virtualbox 4.0 and it seems to be working OK for the most part.
I have however encountered a couple problems.
1. Virtual machines will not start unless the virtual USB controller is disabled.
2. New virtual machine settings get saved in the user folder (C:\Users\)