By Jill Disis

NEW YORK (CNNMoney) -- The Chinese government is refusing to release three activists who were detained while investigating working conditions at a company that made shoes for the Ivanka Trump brand and other prominent retailers.

The U.S. State Department had urged China to free the men immediately and give them "the judicial and fair trial protections to which they are entitled."

But China's Ministry of Foreign Affairs said Tuesday that the matter didn't concern other countries.

"The judicial authorities of China can handle relevant cases according to our own laws and regulations," ministry spokeswoman Hua Chunying told reporters at a regular news briefing. "Other nations have no right to intervene."

The activists went undercover at two factories in southern China, according to China Labor Watch, the New York-based advocacy group they were working for.

China Labor Watch says it has documentation of worker abuse and plans to send a letter with some of the evidence to the Ivanka Trump company.

Li Qiang, the head of China Labor Watch, told CNNMoney that his group hasn't heard anything about two of the men, Li Zhao and Su Heng. The activist group lost contact with them more than a week ago.

A third man, Hua Haifeng, was detained by local police in Jiangxi province, according to his wife.

The Chinese foreign ministry spokeswoman said police put the men under criminal detention because of their use of "illegal monitoring devices." The case is "in process," she said.

China Labor Watch has disputed the allegations against the activists, saying this is the first time in 17 years of probing labor rights violations in China that any of its investigators had been detained by police.

The State Department said it was "concerned" by reports of the detentions.

"Labor activists have been instrumental in helping American companies understand the conditions in their supply chains," Alicia Edwards, a spokeswoman for the bureau of East Asian and Pacific affairs, said in a statement.

The activists took jobs this spring at two factories in Jiangxi and Guangdong that are run by Huajian Group, a Chinese shoe manufacturer. China Labor Watch says the factories made shoes for the Ivanka Trump brand, along with Marc Fisher, Kylie + Kendall, Coach, Alain Delon, Nine West and Karl Lagerfeld.

Su, who was working at the factory in Jiangxi, found evidence of underpaid and overworked employees, according to Li.

Huajian has denied those allegations, though it said last week that it would look into another accusation by China Labor Watch that the company was illegally hiring students as interns.

Huajian spokeswoman Long Shan said Wednesday that the company stopped making Ivanka Trump shoes in March.

Qiang, the head of the group, declined to share with CNNMoney the letter outlining evidence of worker abuse. He said his organization wanted to give the Ivanka Trump company time to respond first.

The evidence purports to show excessively long hours and forged wage slips. He said most of the evidence came from the Jiangxi factory.

The Ivanka Trump company declined to comment Tuesday.

There's nothing to suggest that Ivanka Trump has any connection to what happened to the China Labor Watch investigators. She resigned from her management role at her clothing and accessories company when she took a job in her father's administration, but she still owns part of the business.

Marc Fisher Footwear, which licenses the Ivanka Trump brand for shoes, said last week it would look into China Labor Watch's allegations.

-- Serenitie Wang contributed to this report.