By Taehoon Lee and Joshua Berlinger, CNN

(CNN) -- Chinese authorities have sent the wife of a North Korean defector back to the hermit state, despite an emotional plea from her husband to Chinese President Xi Jinping.

Lee Tae-won told CNN Wednesday that a source inside North Korea had told him that his wife is being held in the border city of Sinuiju for interrogation, after being caught in China.

In October, Lee -- a new name he picked after arriving in South Korea in 2015 -- arranged for his wife and 4-year-old son to be smuggled out of North Korea. They made it out successfully but were later caught in northeast China with eight other defectors, en route to South Korea.

China does not consider those fleeing North Korea to be refugees. It classifies them as "illegal economic migrants," a policy which means they are routinely sent back to North Korea where they are met with almost certain punishment.

When Lee first heard that his wife had been detained in China, he appealed for President Xi to allow her to join him.

But Pastor Kim Sung-eun, a South Korean activist who works to help North Koreans defect and is advising Lee, told CNN a second source inside North Korea confirmed Lee's wife was back inside the country.

CNN is not naming either source or their connections to Lee and Kim for security reasons. It's unclear what happened to Lee's son.

"I will live my entire my life in guilt. It hurts so much and I feel so lonely," Lee told CNN at a demonstration Wednesday outside the Chinese embassy in Seoul. "Because of my decision to bring them to South Korea, my wife will be sent to a political prison and my son will be sent to an orphanage, if he is lucky enough to survive."

Defectors who return to North Korea will often be sent to one of the country's many gulags, known as "kwanliso," sometimes along with their entire families to deter others from trying to leave the country.

The United Nations estimates 120,000 men, women and children are imprisoned in North Korean Gulags.

Lee knows those horrors first hand. He told CNN in an interview published earlier this month he spent five years in North Korean detention from the age of 16 with his father, who died behind bars.

"I feel so empty. I feel so sorry," Lee said at the demonstration. "I want to see them again but I know it will never happen."

South Korea's Foreign ministry would neither confirm nor deny if Lee's family had been returned to North Korea. A Foreign Ministry spokesman told CNN Tuesday that Seoul was attempting to undertake diplomatic efforts to ensure that North Korean defectors will not be forcefully sent back.