Colbert, 49, has been hosting "The Colbert Report" at 11:30 p.m. ET since 2005, in character as a fictional conservative talk-show host. The character will retire with "The Colbert Report."
"Simply being a guest on David Letterman's show has been a highlight of my career," Colbert said. "I never dreamed that I would follow in his footsteps, though everyone in late night follows Dave's lead."
Letterman, who turns 67 on Saturday, announced on his show last week that he would retire sometime in 2015, although he hasn't set a date. CBS said Thursday that creative elements of Colbert's new show, including where it will be based, will be announced later.
Mayors of New York and Los Angeles have already publicly urged the new "Late Show" host to choose their city. New York would appear to have the clear edge, since Colbert is already based in New York and CBS owns the Ed Sullivan Theater, where the "Late Show" has been taped since Letterman took over in 1993.
Letterman offered his endorsement Thursday. "Stephen has always been a real friend to me," he said. "I'm very excited for him, and I'm flattered that CBS chose him. I also happen to know they wanted another guy with glasses."
It's a rapidly changing period for that time slot. Fallon took over for Jay Leno on NBC's "Tonight" show in February, and has dominated the ratings since his arrival, with Letterman and Kimmel running neck-and-neck for second. Chelsea Handler has also said she is about to end her talk show on E! Entertainment Television.
CBS chose not to break the mold: CBS, ABC and NBC will all compete at 11:35 p.m. with shows hosted by white males. CBS, which has an older audience and generally seeks personalities with the widest appeal possible, is taking a chance with a personality whose show has a much more specific appeal. But, like Fallon and Kimmel, Colbert is popular with young men and active on the Internet and social media.
"Our discussions really centered on finding the most talented, the most creative (choice), the person who was going to conduct the most interesting interviews and be the most interesting person himself, and that's what led us to Stephen," said Nina Tassler, CBS entertainment chairman. She said CBS considered several candidates, but did not name them.
Colbert's show won the Emmy for best variety series last year and has earned two Peabody Awards. It's another big move for a Jon Stewart protege: Colbert worked on "The Daily Show" for eight years before getting his own program, and John Oliver is about to launch a weekly show for HBO later this month.
The decision opens up a hole on Comedy Central's schedule. The network said in a statement Thursday that "we look forward to the next eight months of the ground-breaking 'Colbert Report' and wish Stephen the very best."
Stewart told New York magazine on Wednesday night that Colbert would be terrific for Letterman's job. Stewart said he likes what he does and Colbert has a better opportunity to broaden out his comedy than he would.
"He is a uniquely talented individual," Stewart said. "He's wonderful in 'Colbert Report,' but he's got gears he hasn't even shown people yet. He would be remarkable."
The choice of Colbert quickly drew the ire of a real-life conservative talk- show host. Radio's Rush Limbaugh said Thursday that CBS "has just declared war on the heartland of America. No longer is comedy going to be a covert assault on traditional American values, conservatives. Now it's just wide open."
Limbaugh mispronounced Colbert's name as col-bert, instead of the proper pronunciation, col-bear.
The man Colbert is replacing was a target for conservatives, too. Letterman made Republican favorite Sarah Palin a frequent target of his barbs.
Tassler declined comment on what Colbert's ascension will mean for Craig Ferguson, who follows Letterman's show in the 12:35 a.m. time slot and was considered a candidate for Letterman's job.
Colbert would likely enter into some friendly competition with Fallon. Colbert appeared on Fallon's first "Tonight" show, one of a line of personalities in a gag involving people who had to "pay up" on a bet about whether Fallon would ever get the "Tonight" gig.
Brad Adgate, an analyst for Horizon Media, described Colbert as "the best talk-show host available." He said CBS wanted to move quickly to make its choice for Letterman's replacement before next month's meeting with advertisers in New York about the upcoming season's schedule.
"When you've got fire in the belly, you move fast," Tassler said.