In America, white people with an accent - unless it's a few very specific hillbilly or redneck accents - come from somewhere likely to be more civilized than where you are. Accentless English is best found in NorCal and the Pacific Northwest, places that were hinterlands until the 1950s. Even a Philly accent speaks of a culture more refined than yours. Sweden? Ireland? France? Forget about it.

In America, non-white poeple with an accent are upstarts and foreigners from somewhere less civilized. It doesn't matter if Dr. Singh attended Harvard medical school; we're going to hearken back to Apu on the Simpsons because frankly, negative stereotypes are all we know.

The accent matters more than you'd think. I've worked with black producers from the UK and everyone treats them as Brits, not as blacks. A South Indian with a British accent will encounter a great deal less racism than a South Indian with an Indian accent. Gilmore Girls featured a black gay man with a French "gay" accent for seasons; to the best of my knowledge, "race" never figured into the show but "gay" and "french" did. While acknowledging that this is a conscious choice made by a team creating fiction, I think it's also important to note that they were following norms.