When a person who did not got form you what he wnats, tells you negative thigns about yourself, do not be naive. He is whinning (or manipulating) about not getting his interests.
He may call you egoistic, mannerless, etc. all because he did not got what he wants.
Still, do not be naive. These self interested parties may say right things. Just do not take them at face value.
Especially on the moralistic side and that of fairness, remember that your critics are as self interested in criticising as their subject of criticism.
It is funny that people not getting what they want are egoistic enough to accuse with egoism......