Senate Minority Leader Mitch McConnell (R-Ky.) refused to rebuke Rush Limbaugh, Newt Gingrich and other conservative activists Sunday for accusing Supreme Court nominee Sonia Sotomayor of being a racist, despite a growing desire among his colleagues to cool the rhetoric.Appearing on CNNs State of the Union, McConnell repeatedly declined to wade into the growing rift between his Conference and outside activists. While saying that it is certainly not my view, McConnell said, Ive got better things to do than be the speech police.Within hours of President Barack Obamas announcement of Sotomayors nomination, Limbaugh, former Speaker Gingrich (R-Ga.) and others lashed out at the Latina jurist, accusing her of being a reverse racist.The heated rhetoric has caused significant concern among Senate Republicans, who have attempted to take a wait and see approach to the nomination. Several conservative members including National Republican Senatorial Committee Chairman John Cornyn (Texas) and Sen. Orrin Hatch (R-Utah) have publicly condemned the charges.The controversy over conservative attacks on Sotomayor could become a significant distraction for Republicans, who on Sunday were planning to roll out their first wave of concerns with the nomination.As expected, McConnell kept his critiques of Sotomayor to substantive issues related to her work on the 2nd Circuit Court of Appeals particularly on an affirmative action ruling she was involved in earlier this year. McConnell also raised a red flag regarding a 2001 speech Sotomayor gave in which she argued, in part, that she would hope her experience as a Latina could help her be a better judge than a white male.McConnell also refused to take the possibility of filibuster off the table, saying that it is too early to make the call. I think its too early to tell ... [but] the precedent is firmly set, McConnell said, noting that while serving in the Senate, Obama supported a failed filibuster of now-Supreme Court Justice Samuel Alito.In an appearance on ABCs This Week With George Stephanopoulos, Cornyn responded similarly when asked if Republicans would filibuster. I think its really premature to say that or to speculate, Cornyn said, later adding, Im not willing to judge one way or the other.But Sen. Charles Schumer (D-N.Y.), also on This Week, asserted, I think shes virtually filibuster-proof, pending further discussion about the judges record and personal history.Meanwhile, Judiciary Committee member Arlen Specter (Pa.) said Sunday that despite his recent party switch to the Democrats, he feels no pressure to confirm Sotomayor.No, no, Specter said in an appearance on Fox News Sunday, responding to an inquiry about whether he feels bound to support Obamas nominee.I am duty bound under the Constitution to exercise an independent judgment under separation of powers, he added. Specter was one of seven GOP Senators who voted in favor of installing Sotomayor on the court of appeals in 1998.But the Pennsylvania Senator said that he expects the confirmation process could be completed before the scheduled August recess, echoing the White Houses desire to install Sotomayor in time for the September review of which cases will be considered by the court this fall.