By Darran Simon and Janet DiGiacomo, CNN

(CNN) -- Numerous women have accused Hollywood screenwriter and director James Toback of sexual harassment, the Los Angeles Times reported Sunday.

The Times says it interviewed multiple women about their alleged encounters with Toback over several decades; the Times says most of the women spoke on the record.

The article reported that according to the women: "In a hotel room, a movie trailer, a public park, meetings framed as interviews or auditions quickly turned sexual."

CNN has not independently verified the allegations, but is reaching out to the women.

Toback's agent, Jeff Berg, said he would pass a request for a comment on to his client. The director, 72, denied the allegations when contacted by the Times, the newspaper said.

Toback told the paper he had never met any of the women -- or if he did meet them, it "was for five minutes and (he had) no recollection." He said that for the last 22 years, it was "biologically impossible" for him to take part in the behavior the women described in the article, saying he had diabetes and a heart condition requiring medication, the Los Angeles Times reported. He declined to elaborate.

The allegations against Toback come in the wake of a New York Times story this month detailing numerous accusations of sexual harassment against powerful movie producer Harvey Weinstein. The New York Times identified several of Weinstein's accusers.

The allegations against Weinstein sparked the Twitter hashtag #metoo, in which women have shared personal stories of sexual harassment and abuse.

More than 40 women have accused Weinstein of inappropriate behavior, ranging from sexual harassment to rape. Through a spokeswoman, Weinstein has "unequivocally denied" all allegations of rape.

On Sunday, three of Toback's accusers -- Sari Kamin, Starr Rinaldi and Terri Conn -- appeared on NBC's "Nightly News."

None of the women contacted the police at the time of the alleged incidents, the Los Angeles Times reported.

According to his biography on IMDb, Toback has 17 screenplay/writing credits, and 12 credits for directing. His first film, "The Gambler" in 1974, starred James Caan. Toback is also credited with writing and directing "Fingers" in 1978, starring Harvey Keitel and "The Pick-up Artist" in 1987 starring Robert Downey Jr., and Dennis Hopper. In 1992, Toback was nominated for an Academy Award for his role as the screenwriter of "Bugsy" starring Warren Beatty. Toback would later direct Downey in two other films -- "Two Girls and a Guy" in 1997 and "Black & White" in 1999, according to IMDb.

Toback most recently wrote and directed "The Private Life of a Modern Woman" starring Sienna Miller, Alec Baldwin and Charles Grodin, according to IMDb.