Not with a Volvo, but have had similar experiences on vintage cars with 6 volt systems. Weaknesses in electrical systems are exposed early when running only 6V.

Two main starting culprits I've experienced in my 6V cars:

1. Starter drag. When the starter end shaft bearings are worn/out-of-spec, the starter armature will "rock" out of its normal aligned position just enough to drag on the field coil when cranking. More pronounced when motor is hot. Will barely crank, if at all. Cranks normally when cold. Some rebuilt/reman starters are better quality than others, with all components inspected and replaced when out of spec.

2. Connections & Grounds. Particularly with the heavy cables. If corrosion or poor connections are present, high resistance in the circuit shows itself when the motor is hot. Good to check all connections. If the cables are very hot to the touch, there's a problem.