Most of Colbert’s segment was spot on. America’s incarceration habit is expensive. For the last few decades, state and federal governments have put ever more people in prison for ever longer periods of time — and haven’t subjected that policy to cost-benefit analysis.

When governments have tried to save money on criminal justice, as Colbert implies, they’ve traditionally focused on cutting the costs of running the prison — not on the possibility of throwing fewer people in prison to begin with. At its most extreme, that can result in immigration detainees being paid $1/day or less to package meals for inmates at other prisons — or in states charging for public defenders, then putting defendants in prison if they can’t pay their court fees.

But there were two big things Colbert got wrong:

1) Rejecting federal funding to fight prison rape doesn’t save money

One of Colbert’s cost-cutting examples — refusing to comply with the federal standards for the Prison Rape Elimination Act — may not cut costs at all.

Colbert names five states that are refusing to meet the standards, but there are actually seven: Arizona, Florida, Idaho, Indiana, Nebraska, Texas, and Utah. By refusing to work toward the standards, these states aren’t necessarily saving themselves money. In fact, they’re literally costing themselves federal funding.

Colbert says the states are rejecting federal funding for PREA, but that’s not exactly true. They’re actually taking a 5% hit to their existing federal funding for corrections, just for the privilege of refusing to work with the prison-rape standards.

Kristin Hall, the deputy executive director of Just Detention International (an advocacy group that’s worked with the federal government on PREA), says that it shouldn’t cost states that much to implement the standards. “We’ve figured out cost-effective ways to comply with the standards, either reallocating funds or modifying training programs that already exist,” she says.

Hall says that preventing prison rape is ultimately a basic obligation for prisons. But it’s also true that state governments that refuse to cooperate with PREA standards aren’t doing themselves any favors fiscally. “The funding that those states are giving up, essentially, is funding to help survivors of sexual abuse,” says Hall. “We think it’s much more cost-effective in the long run” to comply.

2) Both private and government-run prisons are guilty of exploitation

In a throwaway line, Colbert says that “It cannot be enough to save money. Prisons need to turn a profit.” The implication: the problems with the prison system are because of private prison companies, who are trying to make money off incarcerating human beings. That’s a common sentiment among liberals.

But abuse and exploitation are features of government-run prisons and criminal justice institutions, just as in private ones. Every problem that Colbert describes in the segment is something that both private and government-run prisons are doing (or, as in the case of PREA, something being done by state governments on behalf of government-run and private prisons alike). NPR notes that court fees go to pay the salaries of county employees — and sometimes even their gym costs.

Colbert is absolutely right that the bookkeeping in the criminal justice system is fundamentally backwards: it’s trying to squeeze money out of defendants and inmates once they’re involved in the system, rather than trying to save money by keeping them out of it. But when both the government and private companies are guilty, the problem has to be deeper than profit — it’s a problem with the way that inmates are seen, or not seen, by the public. So segments like this might be the best way to change that.