On the sex/relationship struggle. I feel you. My partner and I have wildly different drives, mine being lower, hers being much higher. She'd happily have sex every night or so, I'm happy with once a week or so. It took some good conversations to get that sorted, because there was the inevitable "Don't you find me attractive?" query, coupled with my personal, and irritating, combination of being very tactile, but not hugely sexual. I like it when she touches me or I touch her, but I don't often have it leading into anything. I just like contact. Rubbing my back is a guaranteed way to win an argument with me, cause I'll just doze right off. She however saw it as an initiation, and so when I stop the contact or start a conversation, she felt it was a rejection.
We compromised, essentially, with her being okay if some times just we focused on her, and other times I'd be in the mood too.
We had some lengthy talks about it, and unfortunately I don't have any advice you haven't already considered, I can only offer understanding. It's much better now, but I know without a doubt she felt similarly towards me, as you do to him. Sick of the rejection, knowing it's not personal but still hurting anyway, thinking about it when we're doing something else.