I'm not sure that's true. Even if it was, it would have to be an attractive Black man with an ugly white woman. Nobody really cares when two physically or mentally ugly people hook up. Being upset only happens when one thinks "a good one" got away.
Anyhow, the only time it really turns my stomach is when I see a Black man with that "look": a certain smugness on his face as he specifically parades his white mate around. IMO, that's usually the problem when people in general get angered by the presence of an interracial couple.