According to Charlotte police, Olsen was charged with felony second-degree forcible rape, cyber stalking, assault on a female, and second-degree forced sex.
Charlotte coach Brad Lambert told Charlotte television station WCNC that Olsen has been suspended from all athletic activities.
The school also said it was investigating the matter.
"This incident occurred a week after a separate, unrelated case of alleged on-campus sexual assault involving acquaintances," the university said in the statement. "Olsen has been suspended from Charlotte athletics, and the University is reviewing the case consistent with its disciplinary procedures."
A spokesman for the Panthers said the older Olsen had no comment and "asks that people respect his privacy."
Olsen's winding and troubled history has included stints at four different schools.
He transferred to FCS school Towson but was also kicked off that team before ever taking a snap for breaking team rules. He had a redemptive 2015 season at Riverside (Calif.) City College before enrolling at Charlotte in 2016, passing for 842 yards and six touchdowns last season.