It's common for men who wear panties to experience feelings of shame or guilt due to societal stigmas and gender norms. It's important to remember that there is nothing inherently wrong or abnormal about wearing panties, and it's okay to enjoy wearing them. The more you do it, the more it just becomes normal to you. Over time you will see them as your panties.
When I'm asked by strangers: "Is that woman's underwear that you're wearing?", my response is usually something like: "Actually I'm wearing my own underwear, but thanks for asking".
__________________
No one ever achieves notoriety by conforming to so-called social norms. With that in mind, just step into your big girl panties and let life happen.
As for me, I have never felt shame about what clothes I wear. Panties feel better than any other underwear, nylon nightgowns to sleep in feel better than any other PJs, women's nylon nightshirts and loungers feel better than anything else to relax in in the evening, so I wear them.
Even my kids are used to seeing me in women's nylon nightgowns and nightshirts. Always have ever since they were young.
There nothing to be ashamed of! Wear what you like and enjoy it. It was different 20 and 30 years ago. Panty wearing is more accepted by society today. We older wearers are the ones who hold onto the stigma that what we're doing is somehow wrong. Let it go.
I can say with myself that early on the shame and guilt of wearing panties and other item of women’s clothing led to me purging.
Purging is such a terrible thing and it will almost definitely lead to you regretting it. Never purge, put you girly stuff away but do not ever get rid of it.
If you are doing things that might be considered 'bad or nasty ', this could be the cause of the problem. If not, I say just enjoy the wonderful feel and fit that you get from the fabric and stop worrying!