I think drilling them out is the way to go.

Chances are the new speakers have a heavier magnet on them than the originals did. The fiberboard door panels really take a beating when you try to hang speakers on them. I am sorry I forget the name of the clips that you can push over the edge of the opening and then put a screw in (speednut??) A full service auto supply store or perhaps a stereo install shop might sell a kit that includes them. The point on the weak fiberboard is to use screws that are long enough in two of the four mounting holes to extend into the metal of the door underneath the door panel. Screw a long screw in before adding the speaker to see where it will hit the door metal and then drill a pilot hole that the screw can enter and securely hold the speaker. If you do this the slamming of the door will not allow the speaker to tear out the fiberboard attachment points. Good luck

Randy