It happened around 2:30 pm Sunday near Highway 99 and the Highway 180 connector, causing a major traffic backup.
Authorities say for some reason, the driver veered off the road and down an embankment before hitting the tree.
All four people were taken to the hospital with major injuries.
CHP officers are now trying to determine why the car lost control.