Stocks of the big public builders retreated on Monday and continued to fall marginally on Tuesday (July 17) after surging Friday on rumors that Warren Buffett, the CEO of Berkshire Hathaway known as the Oracle of Omaha, was mulling a stake in the sector.

Speculation centered around Hovnanian Enterprises, which was up more than 12% in Friday trading. However, other builder stocks were up sharply as well, with 2% to 5% gains. Even beleaguered WCI was up nearly 1%, and Beazer was up more than 4.5%. But on Monday, Hovnanian closed down $0.69 to $17.84, a drop of 3.72%. Beazer and WCI gave back their Friday gains, plus some, in Beazer's case. Most of the builder group stocks, including Hovnanian, were down in afternoon trading on Tuesday, July 17.

Berkshire Hathaway is already invested in the home building business, albeit tangentially, through its stake in USG, maker of gypsum wallboard, and Home Services of America, a real estate brokerage. Buffett is notorious for finding value in stocks that everyone else dismisses.

In response to an email, Hovnanian CFO Larry Sorsby replied that the company had no comment.