The sentencing of former state Sen. Pedro Espada, Jr. took a strange twist Friday when a federal judge gave a detailed accounting of how the judge couldn't have improperly influenced the jury's vote in the case as the once powerful Bronx politician had claimed.

Former state Sen. Pedro Espada is surrounded as he arrives at Brooklyn Federal Court. John Taggart for The Wall Street Journal

Federal Judge Frederic Block then sentenced the politician to five years in prison and ordered him immediately remanded into custody. Mr. Espada, 59 years old, kissed two fingers and waved them at his family as he was led out of the courtroom.

In May 2012, Mr. Espada was convicted of four counts of theft involving the embezzlement of hundreds of thousands of dollars from the Soundview Healthcare Network, a federally funded, not-for-profit hospital and satellite clinics he founded in 1978.

He was accused of spending some of that money on what prosecutors called a "lavish lifestyle" involving vacations, expensive meals, renting a pony for his grandchild's birthday party and leasing luxury cars, including a Bentley for himself. In October, Mr. Espada pleaded guilty to filing false tax returns.

After sentencing Mr. Espada to 60 months, Judge Block said his "normal practice" is "not remanding anyone" in Mr. Espada's position—a first-time offender convicted of nonviolent crimes. He usually allows them to surrender themselves after the Bureau of Prisons picks a prison, he said. That process can take up to a month.

But Judge Block said he determined that Mr. Espada violated his order last week not to have any further contact with the jurors from his case.

"This suggests to me that I cannot trust him," the judge said to Mr. Espada's lawyer.

In a motion earlier this month, Mr. Espada claimed he had evidence of improper communication between the judge and the jury, "and such improper communication influenced a guilty partial verdict."

At a later hearing, Judge Block denied visiting the jury. On Wednesday, Mr. Espada filed an affidavit from juror Luis Ramon. In it, Mr. Ramon stated that on May 14, the jury foreman notified the judge's clerk the jury was deadlocked.

"Within minutes of the deadlock, Judge Block entered the jury room and spoke to all the jurors," the affidavit stated. "Judge Block stated that we should calm down and reach a verdict in this case."

The jury reached a partial verdict that morning finding Mr. Espada guilty of four counts of embezzlement but deadlocking on a fifth theft count.

On Friday, Judge Block dismissed Mr. Espada's allegations. He said there was no note passed from the jury to his clerk regarding a deadlock that morning. He said at 10:35 a.m. the court received a note saying the jury had reached a partial verdict. At 10:37 a.m., the judge said phone records, which he subsequently introduced into evidence Friday, showed his clerk made a call from chambers to the judge's home telephone informing him of the verdict.

The judge said he then drove in from his home in the West Village and entered the Brooklyn federal court underground garage at 11:48 a.m. Swipe card records show he got on the elevator three minutes later and two minutes after that he was in his chambers. At about 12:06 p.m. that day, the judge said, he took the jury's verdict.

"I never realized that everything I do is recorded," the judge said. "I'm glad there is no access card to the men's room."

The judge said he was denying the motion to set aside the verdict. "In light of this evidence that there is simply no way I could have spoken to the jurors between the time they arrived and reached their verdict," he said.

When given a chance to speak before being sentenced, Mr. Espada, who filed the motion without the assistance of his lawyer, told the judge that for strategic reasons he wasn't going to discuss the juror's claims.

Mr. Espada also was ordered to forfeit $368,000 to the government, make $368,000 in restitution to victims and pay $118,000 in back taxes.

Mr. Espada said, "I do look forward to the opportunity to be employed again, to work again and to make amends, mostly with my family."

Judge Block said he was going to refer "what appears to be a false affidavit" to the Brooklyn federal prosecutor's office for "possible prosecution for perjury and obstruction of justice."

Mr. Ramon, who wrote the affidavit that Mr. Espada submitted, was in the courtroom Friday but left soon after Judge Block made those statements. "Nobody told me to lie," he said adding that the trial was a year ago and "I could be mistaken."

Asked if he thought he may be prosecuted, Mr. Ramon said, "I'm concerned but I'm just recalling what occurred in the jury room."

Outside the courthouse, Ms. Lynch said, "of course we'll look into that and do what's appropriate."

Write to Sean Gardiner at