U.S. Treasury yields edged lower Thursday as investors deliberated the Federal Reserve decision on Wednesday to leave rates unchanged.

The yield on the 10-year Treasury fell about a basis point to 4.583%. The 2-year Treasury yield was last at 4.887%, down by 5 basis points.

Yields and prices have an inverted relationship and one basis point equals 0.01%.

Unit labor costs increased 4.7% in the January to March period, the product of a 5% increase in hourly compensation offset by a 0.3% growth rate in productivity, the Labor Department reported Thursday. Economists surveyed by Dow Jones had been looking for a 4% increase in unit labor costs and 0.5% for productivity.

Wall Street continued to digest the latest commentary from the Fed as Chair Jerome Powell indicated it was unlikely that the central bank's next policy move would push rates higher. Policymakers would need to "see persuasive evidence that our policy stance is not sufficiently restrictive to bring inflation sustainably down to 2% over time" in order to hike rates, he said.

Powell also addressed inflation, saying it remained "too high." "Further progress in bringing it down is not assured and the path forward is uncertain," he added.

The Fed on Wednesday also said it would slow the pace of quantitative tightening. Under QT, the central bank is shrinking the size of its balance sheet by allowing maturing bond proceeds to roll off without being reinvested. Effectively, it is a way to restrict monetary conditions.

The new plan will begin in June and will see a reduction of the amount of proceeds from maturing Treasurys that are allowed to roll off the Fed's balance sheet each month.

Investors are looking ahead to the April jobs report set to release Friday. Economists polled by Dow Jones anticipate nonfarm payrolls to have risen by 240,000 last month, compared to a gain of 303,000 in the previous month. The unemployment rate is expected to have held steady at 3.8%.

— CNBC's Jeff Cox contributed to this report.