Volvos have a little wheel to the left of the center air outlet.
This lets in fresh air regardless of the system switches. The
idea is to allow fresh (cold) air to blow on the driver while
everyone else gets warm air (for winter driving). Helps keep the
driver alert (safety, you know). You have to manually close this
input to stop the outside air flow (that also reduces noise, a bit).

Another possibility is the little door that is moved by the vent intake
is not working. I think you can see this if you remove the knee bolstering
on the passenger side and then see if it works. This site has instructions
to remove the bolstering:

http://volvospeed.com/Repair/pdash850.htm