I tried with WinDbg but found it much easier with VC10 debuging executable found at this link "How to: Debug an Executable Not Part of a Visual Studio Solution".
I am still going through the OpenJDK by debuging and found few useful classes, like for example we have used few JVM options like PermSize which are defined in the file "hotspot/src/share/vm/runtime/globals.hpp", here you can find all the global JVM options defined and we have used in our day to day java application.