Just steps from where Congress is debating the fiscal cliff, Rico Ussery, a 26-year-old man who speaks only in monosyllables, cleans restrooms in the Capitol Visitor Center, seemingly unaware of the economic and personal stakes.

He’s one of thousands of disabled workers contracted by federal agencies to stitch flags, serve food and do custodial tasks whose jobs could be eliminated if lawmakers fail to reach a year-end budget deal, triggering steep automatic cuts to defense and other programs.

Concerned about an impasse, advocates for people with disabilities and contractors who employ workers such as Ussery are preparing to salvage a program that drives $3 billion in government contracts a year to nonprofits that employ severely disabled Americans.

Any cuts to discretionary spending would slash $55 billion from the Pentagon’s fiscal 2013 budget. About half of all the projects secured by the government program, AbilityOne, are paid for by the Defense Department budget; the program’s workers have a presence at nearly every U.S. military installation. But Pentagon officials have said operations and maintenance — the very projects most suitable for the disabled — will likely be hit the hardest.

For Peckham Inc., a Lansing, Mich., nonprofit that employs disabled workers to manufacture military clothing, those cuts would mean revenue losses of about 25 percent, or $45 million, the company projects. Eggleston Services Inc., another nonprofit that relies on Navy contracts to employ hundreds of disabled in the Norfolk, Va., area, might have to lay off 20 percent of its workforce.

“We are reluctant to say too much to our employees as these kinds of concern hurt morale and productivity,” said Paul Atkinson, the Eggleston president and CEO, in a phone interview after a financial planning meeting. “We are the infrastructure for cleaning, for feeding, for laundry services in these environments, so when you begin to look at cuts they can happen a lot of ways.”

The White House Office of Management and Budget has not said how much power the Pentagon would have to reshuffle the cuts, which are supposed to affect all programs equally. “We have no idea where sequestration would hit us,” said Bob Chamberlin, the president and CEO of NISH, a nonprofit that acts as a liaison between Washington and 600 contractors dedicated to employing the disabled.

For the past year, lobbyists for major defense firms have outlined their concerns about the impending sequester. The task is much more complex for the beneficiaries of a program for the disabled that, at times, win contracts other for-profit companies would like. The AbilityOne program requires agencies to purchase certain services from contractors that hire the disabled, including those with developmental disorders and physical impairments.

Advocates for the disabled argue that the community suffers more than any other population in a weak economy. They say the program — which pays workers an average of $11.34 an hour — actually saves the government money, increasing payroll tax revenue and reducing reliance on food stamps, Medicaid and other government assistance. NISH says it relies primarily on grass-roots efforts to make its case with lawmakers, organizing annual fly-ins for program participants and inviting lawmakers to visit work sites in their districts.

“The most powerful interaction that we have with Congress are the workers themselves,” Chamberlin said. “That’s the most powerful tool that we have.”

But the organization also has lobbyists, who work a wide range of legislation that affects disabled workers from job training legislation to the defense authorization bill.

“We are explaining to our friends on Capitol Hill that significant cuts in government spending could have a major and disproportionate effect on people with disabilities who rely on government contracts for their livelihood,” said Andy Rosenberg, a lobbyist at Thorn Run Partners, who has represented NISH since early 2010. “It’s a ‘soft’ sort of message, but it is important that we try to make policymakers aware of the human consequences of these negotiations.”

Still, for a routinely underfunded and overlooked community, influencing the debate over such an all-encompassing issue as the sequester seems unlikely. Instead, the nonprofits are eyeing ways to live with tighter budgets, to find efficiencies where they thought no more were possible.

In Norfolk, Atkinson is worried that cuts will not only mean layoffs, but also prohibit his organization from hiring workers with significant disabilities who operate at less-than-full productivity.

“They are not employable out in the general industry because they are not going to meet the productivity standards,” he added. “They got two speeds: slow and stop.”

The AbilityOne Commission, a government entity that manages the program, operates on a budget of more than $5 million. In previous years, the commission unsuccessfully requested more money but didn’t bother in its 2013 proposal.

Rep. Gerald E. Connolly, D-Va., a member of the House subcommittee that oversees the federal workforce, said the cuts would undoubtedly affect AbilityOne.

“That would be, I think, catastrophic,” he said. “It’s really about your values. Do you believe in investing in human capital or not?” he said.

Ussery spent his whole life in special education and said most of his friends are still unemployed. But he is not worried about losing his job. Instead, he’s laser-focused on cleaning seven public bathrooms and maybe becoming a team leader. “I done that so the restrooms keep on getting better and better,” he said.