By Kara Scannell, CNN

(CNN) -- Lawyers for Michael Cohen filed a petition seeking his immediate release from prison, alleging that President Donald Trump's former personal attorney was taken back into custody in retaliation for a tell-all book he is writing about his years as the President's fixer.

In a court filing late Monday that names Attorney General William Barr, the head of the Bureau of Prisons and the warden at the federal prison in Otisville, New York, Cohen's attorneys alleged that the Justice Department retaliated against Cohen for writing a book about the decade he worked for Trump, violating his First Amendment right to free speech. The American Civil Liberties Union joined Cohen in the lawsuit.

"The First Amendment forbids Respondent from imprisoning Mr. Cohen in retaliation for drafting a book about the President and for seeking to publish that book soon," Cohen's lawyers wrote in the court filing.

"The government's effort to exercise prior restraint over Mr. Cohen's book is only the latest in the Trump Administration's efforts to censor speech that reflects negatively on Trump himself or his Administration," it said.

A Justice Department spokesperson declined to comment on the lawsuit.

Cohen's book, which he intends to release in September, will provide "graphic and unflattering details about the President's behavior behind closed doors" including Trump's "pointedly anti-Semitic remarks and virulently racist remarks against such Black leaders as President Barack Obama and Nelson Mandela, neither of whom he viewed as real leaders or as worthy of respect by virtue of their race," Cohen said in additional documents filed with the court in support of his lawsuit.

He said the book will be based on personal anecdotes and "documentary evidence."

Cohen has been held in solitary confinement at the federal prison in Otisville after he was sent back into custody on July 9 when he balked at a provision in his home confinement agreement that would have prohibited his use of social media and engagement with the media, including books.

The filing is the latest in an unusual series of twists surrounding Cohen's sentence since he plead guilty in 2018 to tax fraud, making false statements to Congress, and facilitating hush money payments to two women who alleged affairs with Trump, which the President has denied. Cohen is serving a three-year prison sentence.

In May, Cohen was released from prison partway through his sentence and placed on furlough as part of the Bureau of Prisons handling of the coronavirus pandemic. His lawyers allege the Bureau of Prisons had no issues with Cohen's release for nearly a month until Cohen began tweeting about his book and support for a ruling in favor of a book published by Mary Trump, the President's niece.

Earlier this month, a photographer from the New York Post captured Cohen dining at a restaurant near his apartment with his wife and another couple. Cohen addressed that in the filing saying he was contacted by a prison administrator and told officials in Washington were "upset" with the story. Cohen says he was told he was not in violation of his release.

He said he was advised "it would be best if I remained at home to avoid any further media scrutiny or further upsetting 'Washington'."

On July 9, Cohen and his lawyer Jeffrey Levine went to the federal courthouse in lower Manhattan to transfer his status from furlough to home confinement and be fitted for an electronic monitoring device.

During a meeting with probation officers, Cohen was handed a copy of a Federal Location Monitoring Agreement form that stated as a condition of release Cohen would be prohibited from posting on social media and from engaging with the media -- specifically listing "books."

Cohen objected, and Levine previously told CNN that probation officials said they would check with supervisors to see if that condition could be removed. Cohen's lawyers maintain he never refused to sign the release form. When the officials returned Cohen was shackled by US Marshals.

A spokesman for the Bureau of Prisons previously said that Cohen was taken into custody after he "declined to agree to the terms required for the program and home confinement placement."

Several defense lawyers and legal experts told CNN the no-media provision was unusual.