Should Cart Judicial Reviews be Abolished? Empirically Based Response

Previously published by UK Constitutional Law Blog (21 May 2021).

Empirical study of effectiveness of Cart judicial reviews

The Review and the Government’s Response both relied on cost and effectiveness concerns to recommend that Parliament legislates to overturn Cart.

I encountered three types of complaints about cost and effectiveness of the Cart judicial review procedure (the first two were voiced by the Review):

• there are many Cart claims compared to other JR claims,
• there is a very low rate of positive results brought about by Cart JRs,
• many Cart claims are spurious and thus there is a low rate of permissions granted.

Those complaints are only meaningful in a comparative context. That is, the complaints have bite only if spurious Cart claims occur at a higher rate than comparable non-Cart claims, or if Cart claims have a relatively lower rate of permissions, or a relatively lower rate of positive results. As I show, the picture is much more complicated when viewed in this comparative context and the case against the effectiveness of Cart JRs much harder to sustain than suggested by the Review.

As the Review noted, Ministry of Justice statistics show that Cart JRs have indeed been the single largest topic group among JR claims in the past years. This is correct and I agree that, given this, if the remaining complaints (especially about low rates of positive results) are true, then the case for scraping the Cart procedure is strong.

Are Cart claims spurious at an appreciably higher rate than non-Cart claims? Lord Carnwath, speaking extrajudicially, reported that an anonymous judge of the Administrative Court told him:

I would say that for every 10 days that I sit in the Administrative Court one day is occupied with dealing with spurious Cart applications. The rate of grant of permission for judicial review is minuscule…

To focus on the first sentence from the anonymous judge: it does not follow from it that the rate at which Cart claims are spurious is higher than the rate of spurious claims in non-Cart claims. It could be that the judge was simply more aware of spurious Cart claims because there are more Cart claims than any other single kind of claims. This anecdote and others like it are thus unhelpful for assessing the cost and effectiveness of Cart JRs.

There is, however, a way to measure rates of spurious claims and compare them. A claim for judicial review (including Cart claims) may be denied permission as being “totally without merit” (“TWM”). As I show in the full paper, every year since 2014 Cart JRs have been classified as TWM at a lower rate than (1) all non‑Cart JRs (in the High Court) and (2) those non-Cart JRs (in the High Court) which do not concern immigration and asylum issues. Thus, even though High Court judges may perceive the number of spurious Cart claims as high, Cart claims are being classified as “totally without merit” at a lower rate than other claims.

Regarding the comparative rates of permissions granted, it is true that there is a lower rate of permissions granted in Cart cases: Cart claims have been granted permission at a rate between 6-7% annually since 2017, compared with 17-19% in non-Cart JR claims. However, the difference is (1) arguably not very big and (2) may be explained by the fact that Cart claims have a different, much higher, “second appeals” test for permission. Given the higher bar for permission in Cart cases, we would expect a lower rate. Or in other words, comparing two equally strong claims: Cart and non-Cart, the former could fail to get permission in situations where permission would be granted for the latter. Therefore, the difference in rates of permission also does not provide significant support to the case against effectiveness of Cart JRs.

Finally, the key issue: comparative rates of success (positive results). Relatively few JR claims reach the substantive hearing stage, and it has been widely accepted that a significant proportion of all claims for judicial review that are not refused permission settle in favour of the claimant. My rough estimate is that about a third of non-Cart JR claims lodged in the Administrative Court end with a success for the claimant (see the full paper for detailed analysis and caveats). What about Cart claims?

For the subset of Cart judicial reviews that account for virtually all Cart cases granted permission (i.e. uncontested) the Review defined “positive result” as requiring evidence that the Upper Tribunal (“UT”) in its follow‑up appellate decision concluded that the decision of the First-Tier Tribunal is to be set aside. This definition is reasonable, but only if amended so as to to include cases settled favourably for the claimant as “positive results”.

The Review then “trawled Westlaw and BAILII for all the reports and transcripts of cases involving a Cart JR” (3.41). This was a methodological error, because those databases carry a very small and statistically unrepresentative sample of Upper Tribunal decisions (see the full paper for details). By choosing this methodology, the Review ensured – I am sure unintentionally – that they will miss all but very few “positive results” (as they defined it). Unsurprisingly, the Review found only 12 “positive results” for 2012‑2019. I found that the true number only for claims lodged since 2016 and not including favourable settlements, is at least 71.

The second error the Review made was to calculate the positive result ratio as a proportion of 12 to 5,502, i.e., to what they thought was the total number of Cart applications until 2019. That is, the Review took a result from a very small sample of the relevant data and then used it to calculate a ratio out of all possible cases. Both main errors made by the Review are surprising, but this second one is surprising to an even greater extent. The Review calculated that the rate of positive results of Cart JRs 2012-2019 at 0.22% and stated: “we have concluded that the continued expenditure of judicial resources on considering applications for a Cart JR cannot be defended” (3.46).

In my analysis, I assembled a dataset of over 42,000 decisions of the Upper Tribunal available within the gov.uk domain and created a custom search engine allowing for complex queries of that dataset to identify potential follow-ups to Cart JRs. I then read all the cases identified through the queries for years 2017-2020.

I thus estimated that the true rate of positive results of Cart JRs for 2017‑2019 lies between 2.3% and 9.2%, perhaps close to 7.6% . The maximum of the estimate includes all cases that could have possibly settled favourably to the claimant, whereas the 7.6% figure is adjusted for the rate of  claims that likely settled favourably – following the same method I adopted to estimate rates of success of non-Cart JRs. This is at least over ten times greater than the rate calculated by the Review, but lower than the success rate (including likely favourable settlements) in non-Cart JRs (which I estimated at roughly a third of claims lodged). This estimate is, however, very close to the rate of non-Cart JR claims “successful” in the sense of being allowed at a substantive hearing. I discuss the problems with comparing success rates of Cart and non-Cart JRs in more detail in the full paper. However, I do think that it can be said that the difference in effectiveness between Cart and non-Cart JRs is not such as to give strong support to the recommendation of discontinuing Cart JRs.

How should this affect the Government’s recommendation?

The Government in its Response to the Review explicitly endorsed the aim of the Supreme Court’s rationale in its Cart judgment that this kind of judicial review would provide “for some overall judicial supervision of the decisions of the Upper Tribunal, particularly in relation to refusals of permission to appeal to it, in order to guard against the risk that errors of law of real significance slip through the system” ([50]-[51]). The Response then contrasted this rationale with the Review’s assessment of effectiveness of Cart judicial reviews and concluded that “the Government considers the concept of diverting large amounts of public resources towards these cases to be disproportionate” ([52]).

It should be now clear that the Review’s analysis of effectiveness of Cart JRs cannot stand and that the empirical case for concluding that Cart JRs are a disproportionate burden on public resources is at least not nearly as strong as the Review and the Government thought.

Aside from the effectiveness point, the Government stated that “rendering Upper Tribunal decisions justiciable by Judicial Review is contrary to the intention of Parliament” ([51]). I have considerable sympathy with that argument. 

However, (1) if the Government agrees with the rationale of the Cart judgment and (2) given that the Cart procedure has proven effective at a much larger rate than the Review concluded, then I suggest that the Government can address the main remaining concern (about constitutional impropriety of the Cart judgment) by recommending that Parliament legislates to put Cart JRs on a statutory footing.

This may be also a good opportunity to improve the procedure, for example, by addressing some of the recurring concerns which I have encountered while reading Upper Tribunal decisions (e.g. that decisions to grant permission to a Cart claim do not always give reasons, leaving the UT guessing).

The full empirical study can be downloaded here.