I grew up in the same culture acceptance of these terms, but I’ve grown to think of it like this. If I wouldn’t say it to the group of people it’s describing, using it to describe other people carries the same meaning.

You can’t separate the term from the group of people it’s offending. So using it as an insult reinforces it’s meaning and acknowledging why it’s offensive. So it’s equally wrong.

It’s the same reason I don’t say the n-word or why I wouldn’t use the derogatory terms used for trans people even in a joke.