Deputies say a 13-year-old boy, who is a neighbor, was watching the two-year-old and a six-year-old girl overnight.
Tulare County Fire officials said at least one of the adults who lived at the home was contracted to work the SQF Complex Fires.
It's not clear where the parents were when flames broke out in the Ivanhoe home at around 4:20 on Friday morning.
Firefighters arrived to find heavy smoke and fire coming from the windows on the south side of the home where the bedrooms are located. When firefighters entered the house, they found the 2-year-old boy.
He was declared dead at the scene.
The 13-year-old babysitter and 6-year-old girl were able to escape the house
Officials say the fire damage extended through the entire house.
It's not clear if smoke detectors were activated or working properly.
An investigation is underway to determine what caused this fire to break out. The identity of the child hasn't been released.
EDITOR'S NOTE: The Tulare County sheriff incorrectly identified the victim as a 4-year old initially, but Tulare County Fire officials later updated the age to 2-years-old.