By Heather Long

WAUKESHA, Wis. (CNNMoney) -- Kenneth Olsen lives in House Speaker Paul Ryan's district in Wisconsin. He wishes Ryan had been there to see his wife of 42 years slump in her chair and cry when he told her the news: His GE factory job is being eliminated.

The entire GE factory is moving to Canada.

"She's looking at me, saying, 'what are we going to do?'" Olsen, a soft-spoken 60-year-old said, pausing to brush away tears as he spoke to CNNMoney. "And I'm sitting there going, 'I'm not sure.'"

Olsen earns nearly $30 an hour as a dispatcher at the GE Power plant in Waukesha, Wisconsin. The factory churns out big industrial engines. Olsen is the maestro of the welding department, coordinating where workers and parts go.

"I'm very worried about losing my house," Olsen says. Even if he finds another job, he knows there's little chance he'll earn anywhere near what he earns here. Most companies hiring in the area pay $15 or less.

President Trump has blasted companies for shipping U.S. jobs to Mexico. But Canada is also aggressively luring factories from across the northern border. The Canadian government gave GE $2 billion in incentives to shut down in Wisconsin and move to a city in Ontario, Canada.

It's a huge blow for the town of Waukesha. The engine factory has been a bedrock of the community for over a century. All 350 people working on the factory floor will lose their jobs.

"GE has some of the higher paying manufacturing jobs in the city of Waukesha," says Mayor Shawn Reilly, a Republican.

Should Trump punish GE?

Joe Barlow has labored at the same GE plant for nearly 25 years. He's so mad about the jobs going to Canada that he threw out every GE product in his house, including the toaster and light bulbs.

"I don't own anything GE anymore," Barlow said. "All my family is on board that no one will ever buy another GE product."

Barlow voted for Donald Trump, helping tip Wisconsin red in a presidential election for the first time since 1984. For Barlow, it resonated deeply when Trump promised to put a hefty tax on companies that send jobs overseas.

"I hope [Trump] follows through on his 35% tax and punishes those businesses" that offshore work, Barlow said. He would like to see GE first on that list.

GE CEO Jeff Immelt is a member of President Trump's manufacturing advisory council. Workers at the Waukesha plant, many of whom voted for Trump, are baffled that the president gave Immelt such a prominent role.

Bret Mattice is particularly surprised. He voted for the first time ever in 2016, casting his ballot for Trump because he felt Trump wasn't like the establishment Republicans and CEOs at all.

"Trump's values were different than all of them," says Mattice, another long-time worker at the plant. He believes Trump cares about the "average person."

Mattice gives Trump a B+ grade so far.

GE has begun construction in Canada

Immelt has a history of advising presidents. Before his role with Trump, Immelt was chair of President Obama's Council on Jobs. Obama actually visited the GE Power plant in Waukesha in 2014. He stood in the very place where Mattice and Barlow work every day and proclaimed that it was a model American factory.

No longer. Soon it will be a Canadian plant.

GE has already started construction in Ontario. The company put out a press release calling it a "brilliant factory."

The company's U.S. footprint has shrunk substantially. In 1995, almost 70% of GE's employees were working in the U.S. By the end of 2015, less than 40% of GE's workers were in America, according to a CNNMoney analysis.

GE says the company has "changed dramatically" as it has sold off GE Capital, NBC Universal and its appliance unit. Its business today is more global and industrial. That's why GE bought the Waukesha Engine plant in 2010.

Wisconsin has lost over 125,000 factory jobs

The workers in Wisconsin had hoped Trump would intervene to save their jobs. Their union sent the president a letter on February 6. But now they think their days at GE are numbered. GE has already begun packing up equipment.

"They cut out one piece of your heart at a time. They move jobs out, they move equipment out, they clear space and turn it into stockrooms," says Jeff Neibauer, who's spent nearly a quarter century at the factory.

Neibuaer and his colleagues will soon join a growing list of Wisconsin manufacturing workers who have been tossed aside. The state has shed over 125,000 manufacturing jobs since 2000.

Who's to blame? GE blames the government

GE blames the job losses on Congress.

In the summer of 2015, Congress shut down the Export-Import Bank. It's a government organization that provides money to help boost American exports.

Powerful nations that the U.S. competes against, including China and Germany, have something similar to the Ex-Im Bank. But many Republicans, including Ryan, slammed the bank as "corporate welfare."

"Congress has failed to take action," a GE spokesperson told CNNMoney. "We are the only major economy on the planet without a fully functioning export credit agency."

At the end of 2015, Congress re-authorized the bank. But it hasn't been able to do any major deals above $10 million because the U.S. Senate hasn't confirmed enough people to serve on the bank's board.

Trump used to favor shutting the bank down, but he recently did a U-turn and nominated more people to fill the vacant board seats.

"If Ex-Im had been reauthorized during that time, the decision to close Waukesha would not have been made," a GE spokesperson told CNNMoney.

Ryan says the bank had nothing to do with GE's choice to go to Canada.

"GE's decision to upend the livelihoods of hundreds of Wisconsinites was deeply disappointing," Speaker Ryan's spokeswoman AshLee Strong, told CNNMoney. "The bank was reauthorized in 2015, yet the plant is still closing."

The workers who are about to lose their jobs feel like pawns. They don't believe GE's story -- or their congressman's.

"I would like nothing more than to see Donald Trump come into that planet with Jeffrey Immelt and say, 'This is ground zero for job creation. Let's fill my campaign promise of keeping jobs here,'" says Scott Schmidt, a 44-year-old who thought he'd be at the GE factory for his entire career.

'You can't support a family on $17 an hour'

The GE Power plant itself is located in Rep. Jim Sensenbrenner's district. He's a Republican who has been in Congress since 1979 and is a millionaire. But many of GE's employees live in Speaker Ryan's district next door.

Among the workers, there's a feeling that Ryan and Sensenbrenner are out of touch with middle-class life.

"You can't support a family on $17 an hour," says Joe Acker, a military veteran who still wears a buzz cut and loves making things. The day he got the job at the GE engine factory over a decade ago, he went home and told his wife he was going to work on the "Superbowl of Engines."

The GE jobs pay $30 an hour, almost double Wisconsin's median wage of $17.50. These are the "good jobs" Trump talks about not wanting to lose.

Randy Bryce, a 52-year-old iron worker, is planning to run against Ryan for the congressional seat. It's being called a longshot, but Bryce, who makes $32 an hour, says he hears a lot of anger that "the middle class is under attack in Wisconsin."

What about the workers?

Every GE worker CNNMoney spoke with believed they would be at the factory until retirement. The wages and benefits they earned allowed them to own homes, put kids through college and have the kind of retirement that allows you to buy a camper and tootle around America.

Now they are furiously paying off debt, and warning their kids that they may not be able to afford college.

GE put up notice on the factory floor telling workers not to speak to CNNMoney.

"At no time should employees speak for the company. Nor should they initiative contact with, or respond to, anyone in the media," GE wrote. The notice listed several phone numbers to call. Two were for people outside the U.S., according to a photo of the notice shared with CNNMoney.

GE first made the announcement that it was moving the factory to Canada on September 28, 2015. Employees have had time to prepare and Mayor Reilly says the company is trying to be a good partner and not leave a vacant lot in its wake, but it still stings. The blue collar workers are losing their jobs in Waukesha, but GE is keeping the white collar engineering jobs there.

"Everybody's always trying to (ask), how can we make it cheaper and faster and put more money into CEO's pockets?" Acker says. "We're the ones who are creating the product. We should have a piece of that pie too."

--CNNMoney's Abigail Brooks contributed to this article.