I have these experiences too. I’m autistic and have aphantasia. So my mind is silent, just an empty void, but everything outside of me is too much to cope with due to the sensory issues.

I am SOOOO sensitive to noise too but I think for me it's because my internal dialog/hearing is maybe heightened. I can't visualize anything at all but I can hear and it's so distracting having noise in my brain and outside of it!  

Interesting, I never put those two ideas together.  I definitely find too much activity around me to be stressful and draining, even alarming at times.  For example, going to clubs where the music is loud or where I'm surrounded by a ton of people, is not fun for me, it puts me into a sort of survival mode.  I wrote it off as simply being an introvert, but you may have a point about the experience of excessive outward stimulation being just too much in contrast with my inner experience of quiet.  Thank you for the thought, it is something worth considering.

Yes I have the same problem. I have always been a loner but the main reason is that being around people which I do enjoy, nevertheless I always feel drained after a relatively short time. My wife was always energized by being with people while I was drained, due to the over stimulation. For example, I was overjoyed to have my three kids (all adults and long out on there own) two Sons-in-law and two Grandkids at home this Christmas since it was the first without my wife. Nevertheless, I had to often go up to my room and just get time to myself so I could enjoy being with them again. I had not actually connected this with my aphantasia before, but as you explained it, I realized this was the same reason for me.