In its ruling, the three-judge panel singled out the Democratic governor for ignoring its earlier orders to reduce the state's inmate population to the level ordered by the federal courts.
In January, the governor sought to end the long-running court oversight of California's prison system and promised to take his fight again to the U.S. Supreme Court if necessary. At the time, he decried the billions of dollars California is spending to improve inmate medical and mental health care, saying the court had ordered the state to create "gold plate" prisons that were siphoning money from public schools, colleges, social services and other programs.
The judicial panel was unmoved.
Naming Brown, it said state officials must take whatever steps are necessary to comply with the court's inmate-reduction order by the end of this year or face the consequences.
Otherwise, "they will without further delay be subject to findings of contempt, individually and collectively. We make this observation reluctantly, but with determination that defendants will not be allowed to continue to violate the requirements of the Constitution of the United States."
The governor's office and corrections department did not immediately respond to requests for comment.
Thursday's ruling in favor of the inmates' attorneys was expected and is the latest development in two partially consolidated legal cases that date back years.
One case, filed in 1990, concerns the treatment of mentally inmates while the other, filed in 2001, concerns substandard medical care. In both cases, the federal courts have ruled that the care provided by California's prison system was so poor it violated inmates' constitutional rights against cruel and unusual punishment.
Just last week, one of the judges on the panel rejected Brown's attempt to end the lawsuit over mental health care.
In both cases, the courts have appointed federal overseers to run those aspects of the prison system. The state has responded by building new medical and mental health facilities and hiring hundreds of new employees, many at salaries that exceeding $200,000 a year.
Brown's boldest response has been his so-called prison realignment plan, which took effect in October 2011 and is sending thousands of lower-level offenders to county jails instead of state prisons.
That law alone has reduced the prison population by nearly 25,000 inmates, but that is still not enough to comply with the court order. The state still needs to reduce its inmate population by roughly 9,000 more inmates in its 33 adult prisons by the end of the year.
In January, the state said it had done enough, and Brown said the only way it could meet the court's population cap was to jeopardize public safety by releasing dangerous felons.
The judicial panel acknowledged that Brown's realignment law had made headway, but also said the state had taken no further to comply with the court's directive.
It called the state's long-range prison blueprint "a plan for noncompliance" by allowing the state to house 9,000 more inmates than the court order allows.
"What further steps they will take in order to comply is equally clear: None," the judges wrote.