I notice when I go out to eat alone or are out at an event alone that people tend to stare at me more and the guys especially.
Anyone that is alone at certain events is going to stick out like a sore thumb. It's nature's way of observing someone that one is different.
I've gone to many places alone, and depending on when and where you go, some places you'll be judged very harshly and other places people won't really judge.
A good example is if a person goes out to "hopping" college bars by themself (yes, I've done that a couple times) they will be judged quite a bit. The reason they judge in most cases is because they wonder why I am alone, because almost nobody goes to those type of bars at that time (late night) alone.
I think a lot has to do with timeframe. It's seen as perfectly normal to go to go to a bar to get a coffee or drink by yourself at noon, but it starts getting more weird the later in the day you do it and by nighttime people "hanging out" alone downtown in my city is seen as extremely weird, in fact, it just plain hardly ever happens unless it's some crazy looking homeless person which is more accepted because well, they are seen as crazy to begin with.
The bottom line is society expects people to have friends. So when someone gives off the appearance of not having friends, it will make people stop and stare and talk at times.
I am a male, by the way, and have gotten made fun of quite a bit while by myself. I don't think gender has all that much to do with it.