Former Rep. John Peterson (R-Pa.), who left office in January, said it is time for Sen. Arlen Specter (R-Pa.) to step aside — and Peterson added that he would not support the moderate Republican for re-election in 2010.

Peterson backed Specter in his hard-fought 2004 primary win against then-Rep. Pat Toomey (R). But the former six-term GOP lawmaker told Roll Call recently that the Senator can’t count on his endorsement again next year in his rematch with Toomey.

“If he asks for my support, I will tell him no,— Peterson said in a phone interview.

Peterson commended Specter on his 30-year track record in the Senate, but he said that if the Senator asked for his advice about re-election, he would tell him it was time for he and his wife to “go enjoy your grandchildren.—

Peterson’s remarks come as National Republican Senatorial Committee Chairman John Cornyn (Texas) made plain this week that the committee is backing Specter for re-election and sought to encourage other Republicans to do the same.

“My job as head of the NRSC is to guide the GOP back to a majority in the Senate. I can’t do that without Arlen Specter,— Cornyn wrote in a letter to fellow Republicans. “With him as our nominee, I can target our campaign resources toward beating Democrats and growing the Senate Republican Conference.—

All but two of the Republicans in the Keystone State delegation, including Peterson, threw their support behind Specter instead of their House colleague in the 2004 race. Peterson did press conferences and cut television commercials with Specter, among other things, before Specter went on to win the primary by a mere 17,000 votes.

Peterson said that he and Specter have a long history, starting from when they worked together on the Nixon campaign before either held federal office. Still, he said it was time for Specter to consider stepping down from the Senate.

“I don’t think you’d find me supporting 80-year-olds for re-election,— Peterson said. “It’s not disrespect for him, but there is a time.—

Peterson did not say he would support Toomey in 2010, but said instead that the state Republican party would be well served to have a primary without Specter. He said his endorsement would depend on who else runs in the primary.