When someone does something nice for you, you're supposed to say thank you or acknowledge it in some way.

Whether it be a gift, knowledge or simply holding a door open. That's how manners and courtesy work. When you do not thank someone, it is considered rude. Being annoyed by a rude person is a perfectly normal reaction, and one reason that manners and courtesy became ingrained into our society.

Yes, you should do nice things out of the kindness of your heart. But it's fine to be upset when someone completely ignores the gesture and just keeps on walking. It's one thing to expect tangible repayment, but expecting a "thank you" doesn't make you a bad person. It makes you a normal member of society.

People that think it's okay to be rude will ultimately damage the level of courtesy in society by letting rude people carry on without being called on their bullshit.