The fire broke out around 12:30 a.m. Tuesday at an apartment on 10th Street near E Street.
Neighbors said they were yelling to try to get the family out of the house. A woman, a man, and an infant were able to escape the fire. But, the man went back into the flames when he noticed that two young children were still inside.
Fire officials said the man and two children died in the fire. The woman and an infant inside the apartment were taken to a hospital to be treated for smoke inhalation. They are expected to survive.
"There were multiple victims trapped in the structure. Ultimately there was five people living in this apartment -- two of which, a mother and an infant, have been transported to Mercy Medical Center in Merced. They are undergoing care. I believe their condition is very good," said Mike McLaughlin, Merced Fire Chief. "Unfortunately a father and two children have perished in this fire this morning."
Two other residences were damaged in the fire and the Red Cross was called in to help the people who were displaced.
It is unclear what started the fire.