Figured it out. It's a Bios setting;
Advanced>South Bridge Configuration>Good Night LED>Disabled
When it's enabled, it turns off the power and reset lights. A bit counter-intuitive that it needs to be off, in order for the lights to stay on.
This thread can be closed, thank you.