Housing affordability—or lack thereof—is going to put a damper on the housing market’s growth, according to Doug Duncan, chief economist for Fannie Mae (FNMA) and one of the biggest players in the housing market.

The home market has seen a couple of discouraging signs as of late:

• Two weeks ago, the National Association of Realtors reported that November’s pending home sales number—an indication of future sales—edged 0.9% lower compared to October while year-over-year growth was a modest 2.7%.

• Meanwhile, two ETFs tracking home builders (trading under the symbols XHB and ITB) are close to bear market levels, trading at or near 20% declines from their 52-week highs.

For Duncan, the story for housing is very much tied to whether incomes increase.

Get the Latest Market Data and News with the Yahoo Finance App

“Our theme for the year is [that] housing affordability constrains as the expansion matures,” said Duncan. “Both rents and house prices are appreciating at pretty strong levels. And what is required is to see income growth, particularly at the medium and lower income levels.”

One indicator Duncan has been watching is Fannie Mae’s Home Purchase Sentiment Index (HPSI). That measure of homebuyer attitudes saw a move higher in December, but Duncan is cautious.

“It ticked up a little bit to actually the highest level for the for the time period that we've measured it,” he said, “but it's showing gradual growth in 2016 relative to 2015.”

And while some of the 110 regions Fannie Mae forecasts are showing strong demand—including San Francisco, notably—others are suffering because of their local economies. Though the drop in oil prices is thought of as great for the average consumer, some regions may be facing trouble because of it.

“You see in the oil-related markets some retraction,” Duncan said. “Houston is slowing significantly.”

“So much depends on the ability of incomes to grow,” he added. “We think [2016] will be a better year than 2015, but it won't grow as fast as we saw in 2015.”

More from Yahoo Finance

You've won the Powerball. Now what?

How David Bowie changed finance

Faber: Stocks have been falling for over a year—and it's going to get worse