"How can I know whether the new rack is defective? "

I don't believe it has anything to do with it. The installation is suspect.

Actually, more than suspect, since you found the tie rod end loose.

Once you've changed the ball joints, you are going to need another look at alignment; toe in particularly.

If the toe is not inward, the car will follow every truck rut in the road and feel like it wants to kill you.

Too bad we don't all have Dave to do our work. It seems to me egregious to take money for alignment and miss an untightened tie rod end. Surely there's a good explanation, but I can't imagine what it is. Anyway, this episode reminds me why I do my own work. As the saying goes, at least you know how it was done.
--
Art Benstein near Baltimore

A lot of money is tainted. 'Taint yours and 'taint mine.