Housing inventory is tight across the country and it is affecting potential homebuyers in every category. If you are currently a homeowner who is in a starter home and are looking to move up, it has become significantly more difficult, according to Realtor.com.

The Urban Institute found that the number of repeat home buyers fell from 1.8 million in 2001 to 1 million in 2016. Repeat buyers includes all types of homebuyers: trading up, downsizing and moving to similar-sized homes in another area. However, in 2015, the National Association of Realtors found that only 42 percent of move-up buyers got back into the market, which is down from 55 percent in 2003.

There are a number of factors that come into play, which includes new home construction not being able to keep up with demand, which leads to low inventory that drives up prices. There is also strong competition from investors and foreign buyers who can pay cash. Also, many homeowners do not have enough equity in their current homes to finance a higher mortgage.

“After years of recovery, homeowners are slowly building up equity in their homes again. But it’s not necessarily as much as they need to afford a better home and a bigger mortgage,” says Bing Bai, a research associate at the Urban Institute, to Realtor.com.

Atlanta is a top 10 market where trading up is hardest

Atlanta homeowners especially have a difficult time trading up. According to Realtor.com, Atlanta is the tenth most difficult city to trade up in home. The website looked at a number of factors including the affordability of trade-up homes, measured by percentage of income needed to purchase it, the price gap between starter and trade-up homes, the supply of trade-up homes since 2015 and days on market.

The median price of a starter home in the city is $139,900 while the median price of a trade-up home is $339,000, which is a 242 percent difference.

“In today’s competitive landscape, moving up from [starter homes] to the white picket fence has become increasingly tough,” agrees Javier Vivas, manager of the realtor.com economic research team. “This isn’t particularly unhealthy for the market, since it stops some irrational buyers. But it can leave out a large crowd of buyers who are financially ready.”

This significant gap between an entry-level home and a mid-level home makes trading up financially impossible for many owners. The area of Atlanta also makes a significant difference in the affordability. Realtor.com recommends that homebuyers should try Sandtown if they are on a budget.