By Suzanne Malveaux and Veronica Stracqualursi, CNN

(CNN) -- Sen. Jeff Flake, minutes after he announced he will support Supreme Court nominee Brett Kavanaugh, was confronted at the US Capitol Friday morning by two women who say they are sexual assault survivors.

The Arizona Republican, a key swing vote on the Senate Judiciary Committee, was stopped in a Senate office building elevator on his way to the panel's vote on whether to recommend Kavanaugh and excoriated by two women who wanted to know why he was voting yes.

"What you are doing is allowing someone who actually violated a woman to sit on the Supreme Court. This is not tolerable. You have children in your family. Think about them. I have two children," one of the women, Ana Maria Archila, yelled at Flake. "I cannot imagine that for the next 50 years they will have to have someone in the Supreme Court who has been accused of violating a young girl. What are you doing, sir?"

Kavanaugh has been accused by Christine Blasey Ford of sexually assaulting her decades ago. Kavanaugh has vehemently denied the allegation.

"I was sexually assaulted and nobody believed me," the other woman, Maria Gallagher, told Flake. "I didn't tell anyone and you're telling all women that they don't matter, that they should just stay quiet because if they tell you what happened to them you are going to ignore them. That's what happened to me, and that's what you are telling all women in America, that they don't matter. They should just keep it to themselves because if they have told the truth you're just going to help that man to power anyway."

Flake was visibly uncomfortable. He quietly listened to the women and alternated between making eye contact with them and looking down at the ground.

Gallagher continued through tears, "Don't look away from me. Look at me and tell me that it doesn't matter what happened to me. That you will let people like that go into the highest court of the land and tell everyone what they can do with their bodies."

Flake did not address the two women's questions directly, but repeatedly said, "thank you" -- including when he was asked by Archila if he thinks Kavanaugh is telling the truth.

When Archila insisted on an answer, Flake replied, "I have to go to the hearing."

She pressed him again -- asking, "Do you think that he's telling the truth?" -- to which Flake answered "thank you," again.

He then told a CNN reporter, "I need to go to the hearing. I just issued a statement. I'll be saying more as well. No, there have been a lot of questions here, and I don't want to ask you."

Archila, the co-executive director of the Center for Popular Democracy, and Gallagher, a recent college graduate, had only met that morning and discovered they both were survivors of sexual assault.

The two women were among a larger group of protesters waiting outside Flake's office, and when the senator emerged, the group of protesters and reporters started after him.

As Flake darted into the elevator, Archila caught up, put her foot in the door, and stopped it from closing.

Archila, a New Yorker, later told CNN in an interview that she was sexually assaulted when she was 5 years old. She said she was "profoundly disappointed" in Flake.

"I thought that he was someone who was willing to take a stand, that was on the side of justice, and I thought of him as someone who was able to recognize the humanity across political lines," Archila told CNN. "What he's doing today is saying, 'My party matters more than the stories that I'm hearing.'"

Gallagher, who also saw Flake as a potential ally, told CNN she felt "nauseous" and "upset" upon hearing of the senator's decision. Her plan was to stand in the back of demonstration and hold a sign. But when she heard of Flake's decision, she was overcome with emotion.

Gallagher, 23, told CNN this was the first time she's related her story out loud or publicly.

"It's not something that I ever want to or like to share with people," she said, adding, "But I thought it was important that he knows, and that promoting Brett Kavanaugh is telling victims of sexual assault that no one wants to hear you."