The best reason for being silent instead of getting into an argument is simply to avoid unnecessary conflict so that there’s less suffering. The other person might get mad in the short term, but in the long-term they’ll benefit because you’ve given them less to be resentful about. You might experience discomfort in the short term because part of you really wants to fight back, but in the long term you’ll have less to regret and your emotional state will be more peaceful.
Taking pleasure in someone else getting angry is, from a Buddhist point of view, unskillful. It’s just a subtle form of aggression. Our desire should always be to reduce the amount of suffering our actions cause.