By Emanuella Grinberg and Eliott C. McLaughlin

CNN

AUBURN, Alabama (CNN) -- At least two people were arrested Tuesday at Auburn University amid mostly peaceful protests over the appearance of white nationalist Richard Spencer, whose speaking events have sparked an outcry at other schools.

A crowd of several hundred had gathered by early evening at the public university in eastern Alabama, monitored by dozens of police officers with police dogs. Student groups organized a concert before the speech in what they called a peaceful counter to Spencer's hate speech.

Elsewhere on campus, students told CNN they witnessed a fistfight between a Spencer supporter and a protester that ended with police arresting both men.

Inside, hundreds of people packed Foy Hall, many of whom appeared too old to be traditional students, as Spencer delivered on his reputation for inflammatory rhetoric.

There were several attempts to shout him down as he extolled the virtues of being white and and called on whites to fight for their rights. People called him names and yelled at him to get to his point.

Spencer's supporters occasionally chanted, "Let him speak" when he was interrupted.

Reiterating his key talking points, Spencer denounced diversity as "a way of bringing to an end a nation and a culture" defined by white people.

"There would be no history without us," he said, prompting shouts from the crowd. "The alt-right is really about putting Humpty Dumpty back together again."

Though the auditorium was packed when Spencer began speaking, the crowd thinned considerably toward the end of Spencer's two-hour speech.

Hundreds of students and protesters congregated outside Foy Hall. Students in the crowd reported arguments and plenty of flags and protest placards during the speech but no real violence aside from the fight before the Spencer event.

But as several dozen Spencer supporters exited the auditorium following his remarks, the students and protesters spotted them and gave chase. The mob chased them off campus and through downtown.

Free speech or hate speech?

It was an event that almost did not happen.

Spencer, 38, director of the white nationalist think tank National Policy Institute, has been a target for his radical beliefs. He has advocated a "peaceful ethnic cleansing," where people who are not of European descent voluntarily leave the United States.

After President Donald Trump's election victory last November, Spencer addressed a gathering of the alt-right movement at which he shouted, "Hail Trump!" and audience members apparently gave Nazi salutes. On the day of Trump's inauguration, he was punched in the face by a masked assailant during an interview with the Australian Broadcasting Corporation.

He said his Auburn appearance was sponsored by AltRight.com, a white nationalist site. He paid $700 to rent the hall and an additional fee for security from the Auburn Police Department, according to The Plainsman, an Auburn student newspaper.

Citing safety concerns, Auburn canceled the event Friday. Then, a federal judge granted Spencer's request for an injunction, effectively ordering Auburn to host his speaking event as originally scheduled.

"While Mr. Spencer's beliefs and message are controversial, Auburn presented no evidence that Mr. Spencer advocates violence," U.S. District Court Judge W. Keith Watkins said in his ruling, noting that peaceable free speech is protected by the Constitution.

'Our goal is to not be the next Berkeley'

Debates over free speech on college campuses have flared up in recent months after appearances by such controversial speakers as Charles Murray and Milo Yiannopoulos. A scheduled speech by Yiannopoulos sparked violent protests in February at the University of California-Berkeley.

The injunction prompted Auburn administrators to call for peace Tuesday amid what it called attempts from uninvited, unaffiliated, off-campus groups to provoke racially divisive, disruptive conflict.

"Auburn University supports the rights and privileges afforded by the First Amendment. However, when the tenets of free speech are overshadowed by threats to the safety of our students, faculty and staff, we have a responsibility to protect our campus," the Provost's Office said in a statement.

Last week, the university issued a statement that said in part, "We strongly deplore his views, which run counter to those of this institution. While his event isn't affiliated with the university, Auburn supports the constitutional right to free speech."

But two days later, the university said it was canceling Spencer's visit "based on legitimate concerns and credible evidence that it will jeopardize the safety of students, faculty, staff and visitors."

The challenges seemed to embolden Spencer. That same day, Spencer told his 56,000 Twitter followers he was flying to Auburn and purchasing safety gear.

"They think they have shut this down, but they haven't," Spencer told The Plainsman last week. "I will give a speech on their campus. It is a public place. I think Auburn University is naive and has totally misunderstood who I am if they think that I am going to politely back out of this. I will be there 100 percent."

'Protesting is the best thing we can do'

Jakobi Bailey, a black sophomore from Pelham, Alabama, said he was initially "kind of mad" when he learned that his school had rented space to someone like Spencer, but once he realized that it was public space that anyone could rent, his anger waned.

"People started talking about it, and protesting is the best thing we can do since we can't run him out of here," the 20-year-old said.

Shannon Arthur, 22, a senior had helped organize several minority student groups -- including the Black Student Union, NAACP and Hillel, a Jewish group -- to protest Spencer. But the groups and their leaders were targeted, some by name, on social media before the speech so they opted to join a peaceful protest near the school's football stadium.

They informed the Auburn police that they'd be wearing orange ribbons on their wrists so that authorities could differentiate them in the event of violence, she said.

"You saw Berkeley," Arthur said. "Seeing people really get hurt and bloody is something we don't want for our school population."

Several student groups sought to counter the tension over Spencer's visit by holding an outdoor concert under the hashtag #AuburnUnites.

"I feels like it has less chance of going awry than marching down the street," said Cassidy Kulhanek, a 23-year-old Auburn graduate.

In the shadow of mammoth Jordan-Hare Stadium, bands played in the university green space, which was encompassed in metal police barricades.

Students handed out pizza and shirts with the group's hashtag, which had been made on site with a stencil and bleach.

"We're trying to block hate speech with music and positive ideas and unity," said organizer Jakob Geiger, 19, a sophomore studying political science and English. "We think this is a more effective message to the outside and to our minority students. Our goal is to not be the next Berkeley."

Eliott C. McLaughlin reported from Auburn and Emanuella Grinberg reported and wrote in Atlanta. CNN's Brandon Griggs contributed to this report.