The two government-owned companies are trying to enforce contracts that require banks to take back loans that don't meet certain underwriting standards, Bloomberg News reported.
As of the end of September, banks had not responded to buyback requests worth $13 billion.
About 40% of the requests are eventually withdrawn once the lender provides additional paperwork, according to John Courson, CEO of the Mortgage Bankers Association.
"We're burning a lot of stockholder resources, and clearly a lot of Fannie and Freddie resources, to have 40 percent of these things rescinded," Courson told Bloomberg News. "It hurts the banks and frankly we're wasting government resources, too."