Whats the worst thing/most difficult about being gay?

For me its nearly every friendship iv got/had with a straight male, i always end up trying to show that i in no way have any sexual feelings towards them...just so they know know that i wont be leering or trying anything when weve had a few beers and that.