The 66-year-old financier was found Saturday morning in his cell at the chronically short-staffed Metropolitan Correctional Center, in a unit known for holding notorious prisoners under extremely tight security.
"I was appalled, and indeed the whole department was, and frankly angry to learn of the MCC's failure to adequately secure this prisoner," Barr said at a police conference in New Orleans. "We are now learning of serious irregularities at this facility that are deeply concerning and demand a thorough investigation. The FBI and the office of inspector general are doing just that."
He added: "We will get to the bottom of what happened and there will be accountability."
Barr did not provide specific details but said investigations by the FBI and the Justice Department inspectors general are continuing.
Barr also issued a stern warning, saying the case was far from over. He said anyone who may have conspired with Epstein "should not rest easy."
RELATED: Alleged victims, politicians react to Jeffrey Epstein suicide
[Ads /]
The Medical Examiner called the autopsy of Epstein "inconclusive." Sources had said he hanged himself.
Guards were supposed to check on Epstein every 30 minutes in his cell at Metropolitan Correctional Center. However, sources say that protocol was not followed in the hours leading up to his death.
A source told ABC News that Barr is "livid" and "determined to get to the bottom of this."
"I was appalled to learn that Jeffrey Epstein was found dead early this morning from an apparent suicide while in federal custody," Barr said in a statement. "Mr. Epstein's death raises serious questions that must be answered. In addition to the FBI's investigation, I have consulted with the Inspector General who is opening an investigation into the circumstances of Mr. Epstein's death."
Epstein was in the facility's special housing unit and was on suicide watch when weeks ago he was found unconscious in his cell with apparent self-inflicted marks on his neck.
[Ads /]
The suicide watch ended before his death Saturday. There are many conspiracy theories and questions about why and how this was allowed to happen.
"Was there a crime connected with this suicide? Was someone paid? Was someone else involved?" said Dan Abrams, ABC News.
"It's good that the FBI is investigating, and they're good at this. I want them to do...to leave no stone unturned," Senator Charles Schumer said.
There are also reports of under-staffing conditions at MCC.
[Ads /]
Sources say two guards at the unit where Epstein was housed were both on overtime at the time of his death.
The criminal case against Epstein does not end with his death, law enforcement sources said.
He was the only person charged, so there will be no prosecution. However, the FBI and U.S. Attorney's Office in Manhattan will continue to evaluate the evidence and continue to solicit women's accounts, ABC News has learned.
Several dozen women had come forward since Epstein was charged, and officials had actively solicited accounts from victims.
Epstein's arrest last month launched separate investigations into how authorities handled his case initially when similar charges were first brought against him in Florida more than a decade ago. U.S. Labor Secretary Alexander Acosta resigned last month after coming under fire for overseeing that deal when he was U.S. attorney in Miami.
The Associated Press contributed to this story.