Rep. Mike Castle expected to pack up his Congressional office this year. But he thought he’d be moving across the Capitol Dome to one of the Senate office buildings.

Instead, the Delaware Republican is heading back to his home in Wilmington.

Just a month before his Senate primary election, the nine-term Member was way ahead in the polls and looked certain to sail to the Republican nomination and through the general election into the Senate. But then came a surprising tea-party-backed surge by primary challenger Christine O’Donnell, a political newcomer long thought to be a fluke.

Castle lost, and O’Donnell eventually lost the general election to Democrat Chris Coons.

Recently, from a sofa in his temporary office space for outgoing Members, Castle called the transition out of office “a sad phase.”

“All the sudden it just hits you,” he says. “It’s an unusual end to a way of life.”

Castle recalls his role as one of the chamber’s most prominent moderates. “I never tried to choose who I was working with,” he says. “I didn’t care if it was Republican or Democrat if we were doing something that could advantage people in my state or in my country,” he says.

Castle looks back on his tenure in the House, including why one of his biggest accomplishments has to do with a humble quarter.

On His Regrets About His Campaign

I have regrets about not winning … I think we probably underestimated the intensity of the primary and were too focused on the general election.

We were in good shape for the general election. It was by far the most negative campaign I have been through. There was a lot of outside influence, and by that I mean money brought in through the Tea Party Express group and people like

Sean Hannity and Rush Limbaugh taking me on on a regular basis, even misrepresenting my voting record.

It was extremely unpleasant.

Obviously, we should have focused more on [independents and Democrats], and made an even greater effort to turn them out. I fault myself and my own campaign as much as I do anything else.

On What Role Moderates Can Play

I see a place for moderates in politics in America, and in the Republican party. I think we have to do a better job of articulating where we are and what we are doing. I believe we have to be vociferous in our approaches to legislation.

We’re going to have to create support groups that are going to be able to stand up to some of the outside interests. Huge money was poured into campaigns by very conservative groups … and moderates aren’t up to that at this point. There’s got to be a refocus as far as that’s concerned. … Michael Bloomberg is working with that, and others, and I think you’ll see some shifts and changes.

This election was interesting … The general thinking is that standard Republican candidates, such as myself, could have won [in Delaware, Nevada and Colorado].

Republicans could be in equal numbers [to Democrats] in the U.S. Senate. That should be a lesson for anybody who’s a Republican.

On His Plans for the Future

I don’t know what I’m going to do now. I’ve been talking to law firms in Wilmington and Washington, D.C., about possible employment. My wife says I should just retire.

I will probably not run for office again. That is not part of my future planning. But I have no particular plans or commitments.

I think I counted the books I have backed up to about 62, and I love to read. So I’ll do a little more of that. Jane, my wife, will put me to work if I’m sitting around the house.

On His Relationship With Leadership

The poor Whip’s people, like [Reps.] Roy Blunt [R-Mo.] and Eric Cantor [R-Va.], spent more time with me probably than almost anybody else, trying to say ‘oh, it’s important as Republicans that we do this.’

I feel the political parties assert sometimes too much influence in Washington. They talk about lobbyists, et cetera. But I believe sometimes it’s the party ideological drive that’s too strong.

As a result, I would say no at times. Other times, I would try to bring others with me. … that was never easy … there were times when I just didn’t believe in the direction we were going and I just stood up to it.

On His Biggest Accomplishments in Congress

One was welfare reform. I had worked on that with Bill Clinton … when we were all governors together. After a couple of futile efforts … we were able to get welfare reform done, which I think has been a successful program.

I had another experience that was very different than that, but fun: My second term in office, I was approached by the Coin Collectors of America, and they said they wanted to do a program involving the quarters in which each state would be represented on the quarters. I listened to them for half an hour and said, ‘This is crazy, this is Monopoly money.’

They came back a week later and said, ‘We want to do this in the order in which the states were admitted to the union and Delaware would be first’ — and they said this would make money.

That was fun, and actually made [several billion dollars] for the federal government. I don’t know if I can think of another program in the time I’ve been in Congress that actually made money for the federal government instead of costing money.

His Final Thoughts on His Election

I find it sad for the Republican Party. … Only one [state legislature in the country] had a gain in Democratic seats: it was the Delaware House of Delegates.

Unfortunately, with Christine O’Donnell on top of the ticket, Republicans took a real clobbering in my state.

I’m not a revengeful person. But it’s disconcerting that we could have gone through such a terrible election at a time when Republicans were making gains across America.

I don’t think she served the Republican Party well, and I don’t think she would be a good candidate for the future in Delaware.