It will be interesting to read this brief after it is submitted to the court. Beyond the obvious problem that gay marriage itself is unpopular within the Republican party (only 1/3 in favor, but rising, according to the NY Times article this article pulls from), states rights/federalism arguments resonate with the American Right. They are arguably even more important to the conservative justices on the Supreme Court.
Striking down Proposition 8 as unconstitutional is a significant blow to state power in a realm that is traditionally a state rather than a federal concern (marriage). Depending on the arguments they use, this pro-Big Government stance may be even more radical for these republicans than their pro-gay marriage stance.