Sen. Joe Lieberman said on “Fox News Sunday” that President Barack Obama isn’t responsible for recent scandals at the General Services Administration and Secret Service, but he should be “held accountable for them.”

“The buck stops at the president’s desk,” the Connecticut Independent said. “He’s the leader of our government. He now has to be acting with a kind of relentless determination to find out exactly what happened and to make sure that people who work for him at the Secret Service and GSA and everywhere else in the government don’t let anything like this happen again.”

The GSA is under fire for throwing a lavish Las Vegas conference, and the Secret Service has dismissed six agents and is investigating others over allegations they brought prostitutes to their hotel in Cartagena, Colombia, in what has been a widening scandal.

Lieberman said he has launched an investigation into the culture of the Secret Service to determine whether the actions of agents in Cartagena are an aberration. “We’re going to send them some questions this week asking whether this was an exception,” Lieberman said.

“History is full of cases where enemies have compromised people in security or intelligence positions with sex,” he added.

Lieberman said he was sickened by the scandals and couldn’t comprehend how GSA officials would have believed it was appropriate to incur some of the expenses on the Las Vegas conference.

He also noted that there were warning signs for Jeff Neely, an embattled GSA official at the center of the scandal.

“In 2010 the deputy to Mr. Neely was charged and then convicted of embezzlement. He was embezzling money from the federal government. But that didn’t seem to stop Mr. Neely from the extraordinary waste, fraud and abuse,” Lieberman said.

Asked whether he would vote for Obama or former Gov. Mitt Romney, the presumptive GOP nominee, Lieberman declined to answer.

“I’m going to try to stay out of this one,” Lieberman said. “I think this year when it comes to the presidential election, I’m just going to what most Americans do: go into the booth on Election Day and, in the privacy of the booth, cast my vote.”