The mx server ip will be from the email provider with thousands of whitelisted domains. You get lumped in with the rest and cosidered safe. Spam filters use Bayesian statistics, the logic is not what you expect.