The crash happened shortly before 6pm near Road 236 and Avenue 112.
Authorities believe the 37-year-old may have suffered some sort of medical condition related to Valley Fever.
The car did not go through the home. 15 kids and the homeowners were inside when it happened.
A witness tells the CHP the man was going the speed limit when all of a sudden he veered off to the left and sped up to roughly 80 miles per hour - went 1,500 feet through a field and ultimately crashed into the side of the house.
Alcohol or drugs do not appear to be a factor.