Arizona Sen. Jeff Flake criticized his fellow Republican colleagues for welcoming Alabama Senate candidate Roy Moore in the caucus with open arms.

In a speech on the Senate floor, Flake, who announced he would not seek re-election in 2018, laid into Moore for his previous remarks that Minnesota Rep. Keith Ellison, who is Muslim, should not be allowed to serve in Congress.

“Who will stand today against all cases [of] religious bigotry?” Flake asked.

Flake’s speech came after the Senate Republican caucus welcomed Moore to the Republican Senate luncheon on Tuesday.

“Anyone who says a Muslim shouldn’t serve in Congress shouldn’t serve in Congress,” he said.

Flake also spoke against Democrats’ criticizing Amy Barrett, a nominee for the U.S. Court of Appeals for the Seventh Circuit, for being Catholic.

“I don’t doubt their sincerity, but there is little comfort in their defense that it doesn’t matter that Professor Barrett is a Catholic, but somehow it matters what sort of Catholic she is,” he said.

Flake said he was attempting to stay consistent on his views about religious liberty and spoke to the fact he criticized President Donald Trump’s campaign when he called for a ban on Muslims entering the country.