By Eric Bradner, Phil Mattingly and Dana Bash, CNN

(CNN) -- The Senate could be forced to delay a vote on Brett Kavanaugh's confirmation after Republican Sen. Jeff Flake called for up to a one-week delay so the FBI can investigate sexual assault allegations facing President Donald Trump's Supreme Court nominee.

The Judiciary Committee voted 11-10 on party lines Friday afternoon to advance Kavanaugh's nomination to the Senate floor.

But the vote came after a chaotic scene in which Flake -- who announced his support for Kavanaugh earlier Friday, and then appeared to have a change of heart -- and other senators were involved in an hour-long, behind-the-scenes negotiation about how to proceed.

"I think it would be proper to delay the floor vote for up to but not more than one week," Flake said, so that the FBI can investigate allegations of sexual assault against Kavanaugh. "We ought to do what we can to make sure that we do all due diligence with a nomination this important."

It is not immediately clear whether such an investigation would take place -- or what it would entail. That, Flake said, would be up to Senate Majority Leader Mitch McConnell, R-Kentucky. The committee's chairman, Republican Sen. Chuck Grassley of Iowa, said he would back Flake's request for an FBI investigation, but there is no requirement for the FBI to act.

"This is all gentlemen's and women's agreement," Grassley said after the vote to committee members.

Trump did not immediately oppose the plan that could delay Kavanaugh's confirmation.

"Whatever they think is necessary is OK," he told reporters at the White House.

McConnell has not announced a decision on the request for a delay.

Flake's declaration that he would not vote in favor of Kavanaugh's confirmation "until the FBI has done more investigation than they have already" is significant.

Along with Maine Sen. Susan Collins and Alaska Sen. Lisa Murkowski, he is one of the three Republican swing votes on Kavanaugh's confirmation. If all Democrats and two Republicans oppose Kavanaugh, he cannot be confirmed.

"If (Flake) is joined by one or two other Republicans in that request, then they wouldn't have the votes unless the investigation" is conducted, Minnesota Democratic Sen. Amy Klobuchar said after the committee vote.

Senate vote count

Senate Republican leaders appeared Friday morning to have 49 solid yes votes, one shy of the 50 they need to confirm Kavanaugh -- meaning they could lose one Republican and have Vice President Mike Pence break a potential tie -- so they're going to gamble with a damaged nominee who is viscerally opposed by Democrats.

Friday morning, Flake, presumed to be the swing vote on the committee, said he would back Kavanaugh. It is unclear how his call for an FBI probe would change that.

Two Republicans -- Sens. Susan Collins of Maine and Lisa Murkowski of Alaska -- and two Democrats in red states -- Sens. Joe Manchin of West Virginia and Heidi Heitkamp of North Dakota -- now essentially hold the future of Kavanaugh's nomination in their hands. A third undecided Democrat, Sen. Joe Donnelly of Indiana, announced his opposition to Kavanaugh late Friday morning. Donnelly was one of three Democratic senators who voted for Trump's first Supreme Court nominee Neil Gorsuch.

Flake confronted by two female protesters after announcing he'll back Kavanaugh

Friday's events follow wrenching, partisan hearing Thursday where Christine Blasey Ford detailed her sexual assault allegations against Kavanaugh and he vehemently denied them. It sets into motion days of high drama on Capitol Hill, with the prospect of a conservative Supreme Court for a generation in the balance.

The committee action will be the first step in a series of votes to determine whether conservatives lock in a 5-4 majority on the court. After the committee votes, the GOP's current plan is to hold a procedural vote on the Senate floor midday Saturday and hold the final vote early next week.

Kavanaugh, a Republican who played a lead role in the Ken Starr investigation of Bill Clinton's sexual misconduct and later worked in former President George W. Bush's White House, is a DC Circuit Court of Appeals judge. He would replace the retiring Supreme Court swing vote, Justice Anthony Kennedy, possibly cementing a conservative majority for a generation.

Shortly after the vote time was set by committee Republicans on Friday in a meeting that started at 9:30 a.m. and Judiciary Chairman Chuck Grassley began reading a statement praising Kavanaugh, California Sen. Kamala Harris led several Democrats in walking out of the hearing room. She was joined by Connecticut's Richard Blumenthal, Hawaii's Mazie Hirono, Rhode Island's Sheldon Whitehouse, and later, Vermont Sen. Patrick Leahy.

"From top to bottom this has been about bullies," Harris told reporters outside the committee room. "Dr. Ford came in and she poured out her heart. She cooperated with the process. She gave the process dignity and respect. The least we could do is give her the dignity and respect of a process that has credibility."

"I'm not going to participate in this charade anymore," Hirono said.

Republicans responded angrily, saying they believed Kavanaugh's denial of Ford's allegation.

"I'm a single white male from South Carolina, and I'm told I should just shut up, but I will not shut up," Sen. Lindsey Graham, R-South Carolina, said Friday.

He said he feels sorry for Ford, "but I don't believe it was Brett Kavanaugh" who assaulted her.

"Everything I know about Judge Kavanaugh screams that this didn't happen," Graham said.

Grassley interrupted Sen. Cory Booker, D-New Jersey, when he read statements of Kavanaugh's Yale classmates about his college drinking habits -- including one issued last night saying he'd been lying in Thursday's committee hearing.

Repeatedly, Grassley insisted that Democrats who kept up their calls for an FBI investigation didn't "understand" the committee's own probe.

Democrats also made a last-ditch effort to push Grassley to subpoena Mark Judge -- the man Ford told senators was in the room with Kavanaugh when she says she was assaulted.

As senators delivered speeches in the committee -- Democrats detailing their opposition to Kavanaugh and Republicans saying they support him -- Flake created a stir when he tapped Sen. Chris Coons, D-Delaware, on the shoulder and the two left the room together. It's unclear what they discussed.

Wrenching hearing

During an intense, day-long hearing Thursday, Ford, a California professor, testified that Kavanaugh sexually assaulted her while they were both teenagers in the early 1980s. Kavanaugh later offered a vociferous and emotional defense, alternately shouting and tearing up on national television.

On Thursday, Ford told the committee she is "100%" certain it was Kavanaugh who attacked her at a party when the two were teenagers in 1982.

As the nation watched, she said she "believed he was going to rape me." She told senators it has "haunted me episodically as an adult."

Then, Kavanaugh denied that allegation and other accusations of sexual misconduct he has faced in recent days. He blamed Democrats for what he said was a "calculated and orchestrated political hit" designed to keep him off the Supreme Court. He also refused to support a Democratic push for an FBI investigation of the allegations.

"I've never done this," Kavanaugh said of Ford's assault charge. "It's not who I am. I am innocent."

Late Thursday night, the American Bar Association took the extraordinary step of recommending the Senate Judiciary Committee pause on Kavanaugh's nomination until a FBI probe into the allegations is completed. The association had previously given Kavanaugh a unanimous "well-qualified" rating, its highest rating.

"The basic principles that underscore the Senate's constitutional duty of advice and consent on federal judicial nominees require nothing less than a careful examination of the accusations and facts by the FBI," said Robert Carlson, president of the organization, in a Thursday night letter.

"Each appointment to our nation's Highest Court (as with all others) is simply too important to rush to a vote," Carlson wrote. "Deciding to proceed without conducting additional investigation would not only have a lasting impact on the Senate's reputation, but it will also negatively affect the great trust necessary for the American people to have in the Supreme Court."