By Lisa Respers France, CNN

(CNN) -- Kevin Smith got his big break courtesy of Harvey Weinstein and the famous filmmaker is no longer happy about that.

In the wake of the Weinstein sex scandal, Smith said he will donate the future residuals from the projects he did with the now disgraced mogul to a women's organization.

Weinstein is the subject of allegations from several women who have accused him of misconduct that range from harassment to rape. Through a spokesperson, he has "unequivocally" denied "any allegations of non-consensual sex."

During his "Hollywood Babble-On" podcast with Ralph Garman, which was recorded last Friday and posted Tuesday, Smith said, "My entire career is tied up with the man."

"Everything I did in the beginning has his name on it," Smith said. "And I spent many years lionizing him."

Weinstein's two previous companies, Miramax and The Weinstein Company, produced some of Smith's most notable movies including the 1994 comedy "Clerks," which made him a star.

Smith told the podcast's studio audience that he's not a victim, but he appeared shaken by the controversy.

"It's been a weird fing week," Smith said, his voice at times breaking. "I just wanted to make some fing movies, that's it. That's why I came, that's why I made 'Clerks.'"

"And no fing movie is worth all this," he added." Like, my entire career, f**k it, take it. It's wrapped up in something really fing horrible."

Smith said he feels horrible that he didn't help and "because I sat out there talking about this man like he was a hero, like he was my friend, like he was my father."

So to atone for that Smith has decided to donate his residuals of $2,000 per month to Women In Film, an organization which bills itself as a nonprofit dedicated to promoting diversity in the media arts.

"That feels like a start," Smith said. "Hopefully that goes to people that get to make st without having to deal with some f*ing animal saying, 'Here's the price.'"

CNN has reached out to Women In Film for comment.