Updated: 14 APR 21 10:29 ET

By Kevin Liptak, CNN

(CNN) -- It's an image President Joe Biden kept returning to: the sight of helicopters evacuating Americans from Saigon during the last major battle of the Vietnam War.

In the United States, the image has come to reflect the humiliating end to a prolonged and unpopular foreign conflict. The prospect of overseeing a similar fate in Kabul weighed on Biden as he debated how and when to finally remove the remaining 2,500 US troops in Afghanistan, a person who spoke to him said.

On Wednesday, Biden will formally announce his decision to withdraw all American troops before September 11, the 20th anniversary of the terrorist attacks on the World Trade Center and the Pentagon that led the US into its longest war.

"We cannot continue the cycle of extending or expanding our military presence in Afghanistan hoping to create the ideal conditions for our withdrawal, expecting a different result," Biden will say in his remarks, scheduled for 2:15 p.m. ET on Wednesday.

"I am now the fourth American president to preside over an American troop presence in Afghanistan. Two Republicans. Two Democrats," he will say. "I will not pass this responsibility to a fifth."

In a sign he views his remarks as a historic bookend to the prolonged conflict, he will deliver them from precisely the same spot in the White House Treaty Room that President George W. Bush announced the start of the war on October 7, 2001. Afterward he'll visit the section of Arlington National Cemetery where many of America's war dead from Afghanistan are buried.

Biden will say in his speech that American diplomatic and humanitarian efforts will continue in Afghanistan and would support peace efforts between the Afghan government and the Taliban. But he'll be unequivocal that two decades after it began, the Afghanistan war is ending.

"It is time to end America's longest war. It is time for American troops to come home," he'll say.

The deadline Biden has set is absolute, with no potential for extension based on worsening conditions on the ground. Officials said after two decades of war, it was clear to the President that throwing more time and money at Afghanistan's problems wasn't going to work.

"This is not conditions-based," a senior administration official heavily involved in the deliberations said on Tuesday. "The President has judged that a conditions-based approach, which has been the approach of the past two decades, is a recipe for staying in Afghanistan forever."

Some US troops will remain to protect Americans diplomats, though officials declined to provide a precise number.

Biden has spent months weighing his decision, and determined a war in Afghanistan that killed some 2,300 troops and cost more than $2 trillion no longer fit within the pressing foreign policy concerns of 2021.

That was true of Biden's two most recent predecessors as well, who both attempted withdrawals from Afghanistan only to be drawn back in by devolving security and attempts to prop up the government. Biden has made a different calculation that the US and the world must simply move on.

His foreign policy priorities now lie in Asia, where he hopes to compete with China, and in Russia, whose President he spoke with Tuesday and proposed an upcoming summit.

"We went to Afghanistan because of a horrific attack that happened 20 years ago. That cannot explain why we should remain there in 2021," Biden will say, according to his speech excerpts. "Rather than return to war with the Taliban, we have to focus on the challenges that will determine our standing and reach today and into the years to come."

Still, as Biden was making his decision, the prospect of the Taliban returning to power and potentially rolling back gains on security, democracy and women's rights provided a stark counter-argument to an immediate US withdrawal.

Deliberations stretched longer than some US officials expected, even as Biden signaled repeatedly a May 1 deadline was near-impossible to meet. Hoping to provide space for him to make an informed final decision that he wouldn't come to regret, officials sought to avoid pressuring a President known for blowing past deadlines. Top-level meetings were convened at an unusually high rate.

A long history with the conflict

Officials involved in the process interpreted the lengthy timeline as a sign of Biden's genuine anguish about a path forward, sources said. Biden, meanwhile, made clear he didn't want to be rushed.

In reality, Biden has been thinking about this issue for nearly as long as the war itself, having traveled to the region as a leader on the Senate Foreign Relations Committee and as an internal advocate -- at first ignored -- of drawing down troops during the Obama administration.

On the day in 2001 that Bush addressed the nation from the Treaty Room, Biden appeared on CNN a few hours later from his home in Wilmington. Then the chairman of the Senate Foreign Relations Committee, Biden told Larry King he believed the Taliban would be quickly defeated.

"There is no doubt in my mind the Taliban is done and the American people are going to learn about that and the world is going to learn about that in a matter of weeks, I predict," he said in the interview -- a projection that, 20 years later, appears misguided as his administration works to urge peace talks between the Taliban, which control large swaths of Afghanistan, and the Afghan government.

Still, in the interview, Biden acknowledged the lengthy road ahead -- even though he could not have imagined he would be the president two decades later deciding to pull troops out.

"The hard part is going to be putting it together," he said then. "The easiest part is going to be taking it down."

Over the ensuing decades, Biden would travel to Afghanistan as part of congressional delegations and grill military leaders appearing before his committee.

By the time he became vice president, Biden had adopted a skeptical view toward a continued large presence in the country. Some confidants attributed that to worsening conditions and the growing intractability of the political situation; others said his son Beau's deployment to Iraq as a member of the Delaware Army National Guard lent him new insight into the sacrifices of military families.

At one point in 2009, he hand-wrote a memo to President Barack Obama arguing for a troop withdrawal and faxed it to the White House from his Thanksgiving vacation on Nantucket. He made various attempts to argue his case to Obama, who chose instead to surge troops before eventually pulling many of them out.

Focusing on 2021, not 2001

His views of the war as a drain on American lives and resources haven't changed since then. He campaigned on a pledge to end the Afghanistan war and has reminded advisers in meetings that most Americans have either forgotten about it or say in polls it should end.

Ultimately, he believes the time and energy spent on the Afghanistan war should be redirected to more present-day matters.

"The President deeply believes that in contending with the threats and challenges of 2021, as opposed to those of 2001, we need to be focusing our energy, our resources, our personnel, the time of our foreign policy and national security leadership on those threats and challenges that are most acute for the United States," the senior administration official said, citing as examples China, the pandemic and more disperse terror threats. "Doing that requires us to close the book on a 20-year conflict in Afghanistan and move forward with clear eyes and an effective strategy to protect and defend America's national security interests."

Biden officials said they believed they got a late start in weighing options for Afghanistan because of the delayed transition under President Donald Trump, who refused to acknowledge the results of the election for weeks. Officials said at the time that they weren't able to understand fully the US force posture or assess the entire scope of the agreement the Trump administration had struck with the Taliban.

Starting in February, officials undertook a review of "genuine, realistic options" for Afghanistan, according to an official, mindful of Biden's instructions not to "sugarcoat" the likely outcomes. Heavy consultations between Cabinet members and foreign partners ensued.

During deliberations with senior national security and military officials, Biden chafed at suggestions US troops should remain in Afghanistan for much longer, according to people familiar with the matter, reminding his advisers that -- like his two predecessors -- he promised voters he would end the country's longest war.

There was not unanimous consent among his team. Among those advocating against a withdrawal, Gen. Mark Milley, the chairman of the Joint Chiefs of Staff, had been among the most ardent, suggesting earlier in the deliberations that pulling American troops from Afghanistan could cause the government in Kabul to collapse and prompt backsliding in women's rights, according to people familiar with the conversations.

An annual US intelligence community's assessment released Tuesday was grim on the outlook for Afghanistan, concluding that prospects for a peace deal between the Taliban and the Afghan government "remain low during the next year."

"The Taliban is likely to make gains on the battlefield, and the Afghan Government will struggle to hold the Taliban at bay if the coalition withdraws support," the assessment said.

Biden is fully aware of those risks, officials said, but made his decision based on his firm determination there wasn't a military solution to the various problems plaguing Afghanistan. Still, the images from Saigon linger, even if others reject the comparison.

"This is not Vietnam. This is different," said Chuck Hagel, who earned two Purple Hearts during the Vietnam War before serving alongside Biden as a Republican senator from Nebraska and as Obama's third Secretary of Defense.

"Responsibly managing the end of America's longest war, and the complications of America's interests and helping secure the future for Afghanistan with our allies and the Afghan people is the right course of action," Hagel said.