Assuming that the hand brake isn't on in that position, I think that the hand brake cable on that side might be fully or partially seized. i.e. it's not fully releasing when you let it off. The end of the lever on the rear shoe that is connected to the cable is normally much nearer the lining part of the shoe. It would mean that you've been running round with that brake partially on. There could be something odd about the shoes as one lining is usually shorter than the other. Front one short? Might be a good idea to cross check it with the other side. After a good clean up, a LITTLE brake grease on the rubbing contacts might be a good idea although many people prefer them dry. If they get serviced by the book, very few do, there shouldn't be much build up of dust/dirt/rust.
The "bite" out of the drum will probably be no problem if you file the edge smooth but I'd look out for another one.