Try HWiNFO, HWiNFO32 & HWiNFO64 - Hardware Information and Analysis Tools
You will need to select Sensors Only each time.
Have you tried CoreTemp or RealTemp?
All of these programs are freeware.
**edit** Sorry, I missed the OSD requirement.
MSI AfterBurner has OSD support, but only for selected GPU information.
Maybe there are some windows monitoring gadgets that have an always on top option.