House Speaker Paul D. Ryan returned to the Capitol after meeting with President Donald Trump on Friday as Republican leaders scrambled to get enough votes to pass the GOP-crafted health overhaul bill.

Ryan didn’t use the center entrance to the building — the most direct route to his office — and avoided reporters and TV crews awaiting his return.

The speaker joined a lunch previously scheduled for Trump and Treasury Secretary Steve Mnuchin, White House press secretary Sean Spicer told reporters. Spicer also said despite uncertainty that the bill would pass, he expected a vote at 3:30 p.m.

Amid some reports that Trump is angry at Ryan’s handling of the health care bill, Spicer said the speaker has done “everything he can.”

A Roll Call whip count indicates GOP leadership is fighting an uphill battle to secure the needed 216 GOP votes, with conservatives and moderates balking at the legislation for different reasons. Attempts to assuage each faction have given the other new reasons to oppose the bill.

House Freedom Caucus Chairman Mark Meadows, R-N.C., told reporters Thursday night that leadership and Trump might be 30 to 40 votes shy of their needed threshold. Frantic negotiations have been going on since Trump deployed senior aides to a GOP conference meeting to say Trump insisted on a Friday vote and would simply move on to other issues if the bill died.

White House Principal Deputy Press Secretary Sarah Sanders declined to specify to which agenda item Trump intends to pivot should the GOP health bill fail later Friday. “He wants to get this done and that’s what our focus is on right now,” she told CQ Roll Call.

Lindsey McPherson contributed to this report