Officers pulled over a vehicle just before midnight because they said it was speeding and it didn't have license plates.
The driver initially stopped before taking off. Police said the car then crashed and flipped over.
"The fact that two children lost their lives this morning as a result of really bad decision making is very saddening," Waukesha County Sheriff Eric Severson said.
The driver and a 6-year-old girl were critically injured, police said. A teen and a woman were also hurt.