ESPN's Adam Schefter says Kelly considered joining the Eagles, but in the end decided to remain coaching the Ducks.
While Kelly weighed his options, the Eagles already moved forward.
Owner Jeffrey Lurie, general manager Howie Roseman and team president Don Smolenski left Arizona for Denver on Sunday to continue the search to replace Andy Reid, who was fired after 14 seasons.
They met with McCoy for a few hours and then headed back to Philadelphia. The Eagles plan to interview Indianapolis offensive coordinator Bruce Arians and Seattle defensive coordinator Gus Bradley this week.
The 40-year-old McCoy engineered an offense around quarterback Peyton Manning this season. Denver went 13-3, secured the No. 1 seed in the AFC, and has a bye this weekend. The Broncos ranked fourth in the league in total offense and second in scoring.
McCoy has been successful working with quarterbacks far less talented than Manning. Jake Delhomme went to the Pro Bowl in 2005 when McCoy was Carolina's quarterbacks coach. Kyle Orton had his two best seasons in Denver in 2009-10 under McCoy's tutelage. In 2011, McCoy devised an offensive system around Tim Tebow's skills and the Broncos won a playoff game.
Philadelphia requested permission to interview Cincinnati offensive coordinator Jay Gruden, but the Bengals have yet to grant permission.
The Eagles interviewed Penn State's Bill O'Brien on Thursday and met with Atlanta Falcons defensive coordinator Mike Nolan and special teams coordinator Keith Armstrong on Wednesday. O'Brien is remaining at Penn State.
Syracuse's Doug Marrone was on Philadelphia's list, but he's set to take over at Buffalo for Chan Gailey.
There was plenty of drama surrounding Kelly this weekend. A person familiar with the situation said Kelly was nearing a deal with the Cleveland Browns on Friday before he went ahead with his scheduled meeting with the Eagles on Saturday afternoon.
Kelly's interview with the Eagles lasted several hours, said the person, who spoke on condition of anonymity because the information was not authorized for release.
Kelly also met with Buffalo after his first meeting with Cleveland. In the end, he stayed put just like last year when he nearly accepted a job with Tampa Bay only to change his mind.
The pursuit of Kelly had a soap-opera element because Browns CEO Joe Banner spent the previous 19 seasons in Philadelphia, serving the last 12 as president. Banner and Lurie are longtime friends since childhood. But Banner was forced out of the organization in a power struggle with Roseman and Reid.
Banner had a knack for rubbing people the wrong away in Philadelphia, and wasn't popular in the locker room because of the way he handled contract negotiations. It's unknown whether his reputation was a factor in Kelly's decision not to accept Cleveland's offer.
AP's Rob Maadi contributed to this report.