"I’ve checked/inspected the bulb and it seems to be alright."

"Is this one of those instances where I should Just replace the bulb with a good one and see if it works?"

I'm having a hard time reconciling these two statements. If you've checked/inspected the bulb and it's ok, why would you need to replace it with a known good one to test further?

It should illuminate with ignition on/engine off -- and then go out when you start the car if all is well with the hydraulics of the brakes. The way you inspect the bulb to see if it's actually alright is -- 1) pull it and put power/ground to it to see if it illuminates; or 2) replace it with a known good bulb and see if it illuminates. If you haven't done one of those two things, then you haven't "....checked/inspected the bulb...." to determine if the bulb is faulty or if it's some other problem.