Bad news for National Security Agency critics who want to use the annual Defense authorization bill to exact changes to the agency’s spying policies: Speaker John A. Boehner doesn’t think the defense measure is an appropriate vehicle for that debate.

“I don’t know that in the National Defense Authorization bill that that issue ought to be done,” Boehner told reporters Thursday. “It ought to be done on its own.”

Boehner teamed up earlier this year with Democratic leadership and the White House to narrowly defeat an amendment from Rep. Justin Amash, R-Mich., that would have ended the NSA blanket collection of telephone metadata. That amendment was attached to the defense appropriations bill in July after Amash and a band of libertarian-minded Republicans looked poised to vote down the rule governing floor debate for the Pentagon spending bill if they didn’t get a vote on the NSA amendment.

Even though the speaker rarely votes, Boehner voted against the Amash amendment.

On Thursday, the Ohio Republican once again defended the NSA, saying the agency “protects the American people, protects, frankly, our allies around the world.”

Boehner did, however, signal that he thinks there needs to be changes to some NSA policies and programs.

“Yes, I think there are changes that need to be made,” he said. “But they need to be made in a very thoughtful way.”

If, as expected, the Senate passes the defense authorization measure, the House and Senate will need to conference their differing versions of the bill — and the legislation will come up in the House as a vote on the conference report. While NSA policies remain a hot-button issue, it’s rare that amendments are allowed on conference reports. It will more than likely be an up-or-down vote on the yearly defense authorization bill, which has been signed by the president for 51 consecutive years.

NSA critics have a better chance of changing NSA policies when the Senate bill comes to the floor — and hoping that those changes can survive a House and Senate conference.

The NSA has been embroiled in a string of controversies regarding its intelligence collection since former national security staffer Edward Snowden blew the lid off a massive phone data collection program. It was later revealed that the NSA was collecting internet user data and conducting phone surveillance on foreign leaders, such as German Chancellor Angela Merkel.