Donald Trump has been trolling Hispanic and Latino voters from the moment his campaign began. A lot of this has had to do with the patronizing way he refers to them. In a Republican debate in Nevada earlier this year, he declared himself “number one with Hispanics.” He invokes that love for “the Hispanics” often, including in his infamous Cinco de Mayo taco-bowl tweet. If one of those Hispanics or Latinos does something he doesn’t like, then Trump gets more direct with his insults.

Last Friday, the same day a federal judge released documents showing how much of a scam the now-defunct Trump University really was, the presumptive Republican nominee made sure we all knew that the judge is “Mexican.” (Judge Gonzalo Curiel’s parents were Mexican immigrants, but Curiel was born and raised in Indiana. Trump clearly meant it as a slur.)

Trump has repeatedly torn into Hispanic and Latino-Americans, the very folks Republicans will need the most to thrive. In March, Gallup showed that 77 percent of Hispanic voters view Trump unfavorably. That’s nearly the percentage by which Hispanic voters went for President Obama in the 2012 election. Back then, Republicans published a much-mocked “autopsy” report that declared, “If Hispanics think we do not want them here, they will close their ears to our policies.”

According to a recent BuzzFeed report about the concerns of many Hispanic Republicans regarding Trump, operatives in charge of cultural outreach at the Republican National Committee are worried that he’ll reverse any progress they’ve made with Hispanic and Latino voters, and other officials and Bush White House veterans are fretting about the candidate’s tone. On Wednesday, Ruth Guerra, the head of Hispanic media relations at the RNC, announced that she’s resigning, reportedly out of discomfort with telling her community to vote for Trump. (Guerra, who is of Mexican descent, told the New York Times that she is joining a Republican super PAC, the American Action Network.)

It’s healthy to see Guerra and other Republicans taking a stand against Trump’s cartoonish racism. But it’s one thing for party members to understand that they can’t afford to alienate Hispanic and Latino voters, and quite another for them to make it seem like Trump is the sole reason they are losing ground with those voters. Sure, he’s terrible. But rather than this moment being a reckoning for Republicans and their willful undermining of Hispanic and Latino families for decades now, I fear that the party, after he loses in November, will try to get away with making their nominee a bogeyman. It’ll later say, “That was him, not us!” Damage done by Trump’s political rhetoric, after all, is more easily addressed than that done long ago by policy.

The BuzzFeed report and Guerra’s subsequent announcement were hardly the first time party voices have gotten loud about Trump, concerned that he’ll make them look bad. Ever since he called Mexican immigrants “killers” and “rapists” in the speech that launched his presidential campaign last June, Trump has had a few Republicans publicly worried that his candidacy will bury them long-term with Hispanic and Latino voters. It would be smart for them, politically speaking, to make that bloc a priority for the party even over black and Asian voters, two other growing constituencies that lean heavily Democratic. The browning of America is well under way, and Hispanic and Latino populations are leading that growth.

I despise the term “majority-minority” (it suggests that white people are being replaced as the dominant racial group), but it’s likely the term you’ve heard the most to describe our cultural metamorphosis. The U.S. Census stated that, by 2044, “no group will have a majority share of the total and the United States will become a ‘plurality’ of racial and ethnic groups.” 2020 is the census estimate for when “more than half of the nation's children are expected to be part of a minority race or ethnic group.” Hispanic and Latino populations are expected to see the most growth of all. More than half of U.S.-born Latinos are millennials. That’s according to Pew, which also reported that millennials make up 44 percent of the Hispanic voting bloc — the highest percentage of any ethnicity. That’s nearly half of the record number of eligible Hispanic voters in 2016 (27.3 million). And as Republicans realize, those folks are mostly Democrats.

It’s no surprise, then, that Trump’s thinly veiled white nationalism and outlandish ideas about immigration are so popular with so many white Americans. Even he has to know that Mexico would never pay for a border wall, and that he couldn’t marshal the forces to round up 11 million undocumented immigrants. But he keeps saying he’ll make these things happen. “Make America Great Again” is less a dog whistle than a bullhorn, announcing to every marginalized community that it’s time to go back to the good old days for straight white guys with money.

As Trump’s immigration rhetoric makes headlines, Hispanic and Latino voters worry about the same stuff most of us do. In the summer of 2012, registered Hispanic voters listed health care, unemployment, and economic growth as their top priorities in a Gallup poll; only 12 percent listed immigration as such. Pew found similar results in 2014.

Hispanic and Latino voters haven’t shunned the Republican Party simply because its elected officials talked badly about their communities. They’ve left because the party has made it a priority to undermine them. Nearly 24 percent of Hispanic families live in poverty, and the GOP’s regressive economic policies will only serve to deepen that poverty, the same way that they do for black and other communities of color: by opening avenues to wealth-building for those already doing well. Republican-led erosion of union power hurts Latino workers, too, who make up the third-largest union membership in the United States.

Another problem with the Trump-ruined-us-with-Latinos line Republicans offer is that they’ve already ruined their relationship with those voters via legislation written, passed, and signed into law by their own GOP comrades. Just look at California’s Proposition 187, which was aimed at barring undocumented immigrants from receiving any taxpayer-funded services. It was passed in 1994 and later ruled unconstitutional, but the damage was done for Republicans in the state; Obama received 78 percent of the Latino vote there in 2012.

A Los Angeles Times report this week detailed how current California Republicans see Trump having the same effect that Prop 187 had, but it’s doubtful he could make things much worse for the party than they already are. That goes for the country, too.

If Republicans want to survive the browning of America, they’ll need more than some token outreach and improved rhetoric. Trump may say some awful stuff, and that stuff may turn off a lot of people who would otherwise have been willing to listen to Republican ideas. But Trump is hardly the first offensive, unserious Republican candidate to have run for president. And the sooner Republicans stop blaming Trump for failings of outreach that came before he said a word, the better off we’ll all be.