By Nathaniel Meyersohn, CNN Business

(CNN) -- Retailers and restaurants say eliminating cash at their stores makes them more efficient. But opponents argue that cashless stores exclude millions of Americans without bank accounts.

Americans use cash in 30% of all transactions, according to a 2017 survey from the Federal Reserve Bank of San Francisco. People dish out cash for most purchases under $10. And cash is also still the most common form of payment for people making less than $25,000 a year.

Now, cities and states are starting to take action against cash-free stores.

New Jersey Governor Phil Murphy signed a law last week banning cashless stores. Philadelphia also enacted legislation prohibiting cashless stores earlier this month, and officials in New York City, Washington and San Francisco are considering similar legislation. The federal government does not require retailers to accept cash.

"While card-only may be convenient for some businesses, it can actually be discriminatory against poor communities that don't have as much access to banks or lines of credit," said New Jersey State Senator Nellie Pou, who sponsored the state's new law.

The 'burden' of cash

But Amazon Go stores, Sweetgreen, Dig Inn, Dos Toros Taqueria and local stores around the country can't seem to ditch cash fast enough.

Chains like Sweetgreen, which eliminated cash at stores in 2017, argue that going cashless is safer, cleaner for employees and speedier at stores. Sweetgreen has noted that most airlines have eliminated cash and some countries, like Sweden, have shifted away from cash.

Employees at Sweetgreen spent two hours a day counting bills, but eliminating cash helped reduce long lines and increased the number of transactions per hour, the company said. It also allowed Sweetgreen to save money on paying for armored cars to pick up and transport cash.

"Welcome to the future, baby," Sweetgreen said.

Michelle Gauther, founder of Mulberry & Vine fast-casual restaurants, told the New York City Council that the "management of cash can be an incredible burden." Her chain went cashless in 2016, and it has been "a huge win for my employees."

Other businesses have eliminated cash for similar reasons. Even sports stadiums are starting to jump in on the cashless trend.

Mercedes-Benz Stadium, home to the NFL's Atlanta Falcons and MLS's Atlanta United, became the first pro sports stadium to go cashless earlier this month. The stadium said its decision will help it cut down wait times and give it flexibility to lower prices more often. It set up 10 kiosks around the stadium where fans can convert their cash into pre-paid debit cards to buy food and other merchandise.

Credit card companies, which collect fees from businesses when customers make purchases, are proponents of the cashless movement. Visa even offered restaurants $10,000 in 2017 for its "Visa Cashless Challenge" if they pledged to stop accepting cash.

America's unbanked

Opponents argue that cashless stores leave out low-income residents without credit cards or bank accounts. Eliminating cash disproportionately hurts minorities, immigrants and senior citizens, critics say.

Around 8.4 million US households, or 6.5% of the country, were "unbanked" in 2017 — meaning they did not have a checking or savings account— according to the most recent FDIC data. Another 24.2 million households, or close to 19%, were "underbanked," meaning they had a bank account but also used other financial services like money orders, check cashing or payday loans.

Although the number of minorities and younger households without bank accounts has fallen in recent years as the economy improves, both remain above the national average. Close to 17% of black households and 14% of Hispanic households did not have a bank account in 2017, according to the FDIC.

There are several reasons why Americans don't have bank accounts beyond not having enough money to keep in them. A third of unbanked households say they don't trust banks, the FDIC found in a survey. Others say they avoid banks because of privacy concerns and because bank account fees are too high or unpredictable.

Dos Toros founder Leo Kremer expressed sympathy for unbanked citizens in a letter to the New York City Council. Although he said New York's proposed cashless legislation was "well-intentioned," he called it "ill-advised and burdensome regulation."

"Let's pull those members of our community forward into the modern financial system, rather than pulling the business community backward," Kremer said.

'Seemed unfair'

Cities are trying to balance bringing in new technology while also supporting disadvantaged citizens.

In Philadelphia, the poverty rate hovers around 26%. Mayor Jim Kenney signed legislation banning cashless stores "despite our continued concerns about how this legislation might impact innovation in our retail sector," said city spokesperson Mike Dunn. "We will continue to monitor this, as we face the ongoing challenge of growing our economy while ensuring that growth is inclusive."

Bill Greenlee, the Philadelphia City Council member who spearheaded the effort to ban cashless stores, said he was bothered that coffee shops in downtown Philadelphia refused to take cash for small purchases, even though they accepted cards.

"It was creating an us and them kind of situation," he said. "It just seemed unfair."

The Philadelphia law will go into effect in July and it excludes online sellers, parking garages and wholesale clubs. It also includes an exemption for "transactions at retail stores selling consumer goods exclusively through a membership model." That provision was designed to allay Amazon's concerns about the legislation, said Lauren Cox, spokesperson for the city's Commerce Department.

But Amazon still doesn't feel the exclusion applies to its stores, she said. That's because you only need an Amazon account to shop at Amazon Go cash-free stores, not a Prime membership. A spokesperson for Amazon declined comment.

Greenlee said that city officials wanted to act quickly "before this phenomenon grew and we were really inconveniencing more businesses" by forcing them to switch back to taking cash.

But business advocates, including the Pennsylvania Restaurant & Lodging Association, are concerned that the law will drive away companies.

Greenlee, however, does not expect the legislation to hurt Philadelphia's economy or discourage businesses from setting up shop in the city in the future.

"We're not asking anybody to reinvent the wheel here," he said. "We're just asking them to do what businesses have been doing for centuries."