The Republican-controlled House unexpectedly rejected short-term funding for the Department of Homeland Security on Friday, increasing the prospect of a partial shutdown at midnight of an agency with significant anti-terrorism responsibilities.