By Paul R. La Monica, CNN Business

(CNN) -- The world's most well-known investor says it's hard for him to do better than the broader market. So should you even try?

Warren Buffett told CNBC on Monday that he's had a "tough time" trying to beat the S&P 500. The Oracle of Omaha, who just released his annual Berkshire Hathaway shareholder letter, suggested that the index is still the best way to invest in the stock market for most people. He even joked that most of the time he doesn't know how to pick individual stocks.

Buffett told the network that his two investing gurus, Ted Weschler and Todd Combs, have each underperformed the S&P 500 during the past few years by a "tiny bit." Even so, he added that their stock picks have done better than his.

Of course, the long-term track record of the world's third-richest man says otherwise. But he has a point.

Taking less risk

It seems silly for investors to try picking and choosing the best stocks — the market, writ large, historically tends to go higher. Just ride the wave, and you'll do fine.

But why the S&P 500 and not the Dow Jones Industrial Average? The S&P 500 may not be as famous as the Dow. But this blue chip index has exposure to nearly all of America's most important companies, and it ranks them by how large they are.

That makes it a much better proxy for the US economy than the Dow, which has only 30 stocks in it and weighs them by their stock prices.

Unsurprisingly, the S&P 500 now skews toward technology. The sector has done extraordinarily well over the past decade because of growth in online advertising, social networks, e-commerce, cloud computing and mobile gadgets.

The four most valuable companies in the S&P 500 are Microsoft, Apple, Amazon and Google owner Alphabet. Facebook ranks sixth. The only non-tech company among this group of giants: Berkshire Hathaway, which is in fifth place.

Berkshire Hathaway is famously underweight the technology sector, however.

Yes, Buffett's top holding is now Apple. But before Berkshire invested in the company, its favorite tech stock was IBM — not exactly the most exciting name in the industry.

Most of Berkshire's other top stocks are far less dynamic than Apple, too. Big financial firms Bank of America, Wells Fargo and American Express rank among its five largest investments.

Buffett is also a fan of consumer staples Coca-Cola and struggling Kraft Heinz. And his firm has recently fallen in love with airlines: Southwest, Delta, United Continental and American are among Berkshire's top 20 holdings.

All of that means it should not be a major surprise that the Berkshire portfolio has lagged the market lately.

Better than bonds

Instead of looking at Berkshire, investors should focus on what Buffett thinks about the stock market broadly. He noted that the market is cheap, especially with longer-term US Treasury bond yields like the 30-year hovering around 3%.

Buffett told CNBC that faced with a choice between bonds and stocks, the decision is easy.

"I'd buy the S&P in a second," Buffett said, adding that he would give "enormous odds" that the S&P 500 will do better than 30-year bonds over the next three decades.