If we use chromosomes, what about a person who was born intersex, and does not have a distinct XX or XY? Born XY? Sure, rule them out. Born with an indeterminate sex? Testosterone levels must be sufficiently low.
Besides, Could such a person sweep after lowering their testosterone levels for a continuous period of six months or longer? I'm not so sure. At an elite level I'd think that would make an enormous difference.