However, it is anti-gay discriminatory behavior. Someone might in their heart-of-hearts not feel the least bit anti-gay, but fail to recognize that denying marriage equality is in fact, anti-gay behavior.
You see this all the time in Detroit. White people from the suburbs who know without a doubt that they aren't anti-black, fail to treat blacks as their equals. Talk is talk. Ignorance does not absolve you of the consequence of your actions. You can harbor no ill-will towards homosexuals, and yet treat them in an inequitable manner. If so, your actions, are anti-gay. You are functionally anti-gay.
If by "being against gay marriage" you mean "against gays having the same rights as non-gays" then yes, it's homophobic.