What Happened?
In the recent decision of [1] Ardestani v. Canada (Citizenship and Immigration) 2023 FC 874, part of a tetrology of cases where Federal Court justices were critical over attacks on IRCC’s Chinook system, Justice Aylen did not mince words.
She writes:
II. Preliminary Issue
[8] At the commencement of the hearing, counsel for the Applicant advised that he was relying on his written representations but requested that counsel for the Respondent answer five questions related to this matter. As I advised counsel for the Applicant, a hearing of an application for judicial review is not an examination for discovery. Counsel for the Respondent was under no obligation to answer his questions. Moreover, it was not open to the Applicant to raise new issues at the hearing of the application.
[9] I also raised with counsel for the Applicant the fact that two decision of this Court have recently been issued – Raja v Canada (Minister of Citizenship and Immigration), 2023 FC 719 and Haghshenas v Canada (Minister of Citizenship and Immigration), 2023 FC 464 – in which counsel for the Applicant made a number of the same arguments as raised in this application and which were all dismissed by this Court, twice. I asked counsel for the Applicant if he was continuing to pursue these issues notwithstanding the earlier findings of this Court and he indicated that he was.
[10] I find that counsel for the Applicant’s attempt to re-litigate such issues and to transform the hearing of this application into an examination for discovery constitutes an abuse of this Court’s processes. (emphasis added)
In the decision, Justie Aylen comments about the arguments made by applicant against Chinook:
[26] The Applicant asserts that his work permit application was processed using Chinook, which in and of itself is a breach of procedural fairness. Moreover, he asserts that the use of Chinook was improper given the importance of the decision at issue and the degree of complexity of the decision at issue (which involved business immigration). There is also no merit to these assertions. I am not satisfied that the use of Chinook, on its own, constitutes a breach of procedural fairness or that the nature of the application itself has any bearing on the use of Chinook. The evidence before the Court is that the decision was made by an Officer, with the assistance of Chinook. Whether or not there has been a breach of procedural fairness will turn on the particular facts of the case, with reference to the procedure that was followed and the reasons for decision [see Haghshenas, supra].
….
[34] The Applicant further asserts that the use of Chinook is
“concerning”, suggesting essentially that any decision rendered in which Chinook was used cannot be reasonable. I see no merit to this suggestion. The burden rests on the Applicant to demonstrate that the decision itself lacks transparency, intelligibility and/or justification, and baseless musings about how Chinook was developed and operates does not, on its own, meet that threshold. (emphasis added)
While Justice Aylen discusses two other cases Raja and Hagshenas, there was also a third that was released just before Ardestani, in Zargar. All of these cases involve essentially an identical fact pattern of Iranian C11 applicants being refused work permits.
For the interests of summarizing the discuss of Chinook on each, and notwithstanding that I have only seen the full file record in Hagshenas (and have also written a past post, see: here), I will extract block quotes of what judges have said in each of the remaining three decisions about Chinook.
[2] Zargar v. Canada (Citizenship and Immigration), 2023 FC 905 (CanLII), <https://canlii.ca/t/jxxpc> – Justice McDonald, Dismissed
[12] Firstly, the allegations regarding: (a) the use of Chinook, (b) reasons only being provided after the judicial review Application was filed, and (c) the length of the processing time were fully canvassed in both Haghshenas v Canada (Citizenship and Immigration), 2023 FC 464 [Haghshenas] and Raja v Canada (Citizenship and Immigration), 2023 FC 719 [Raja]. In the absence of any specific evidence to support these allegations in this case, I adopt the analysis from those cases (Haghshenas paras 22-25, 28; Raja at paras 28-38) and can likewise conclude that the Applicant has not established any breach of procedural fairness on these grounds. (emphasis added)
Note that there was an apparent lack of evidence filed in Zargar.
[3] Raja v. Canada (Citizenship and Immigration), 2023 FC 719 (CanLII), <https://canlii.ca/t/jxfdq> – Justice Ahmed, Dismissed
[24] The Applicant submits that the Officer assessed his work permit application on the basis of irrelevant and extraneous criteria, but does not specify which criteria. The Applicant also submits that the IRCC’s reliance on Chinook, an efficiency-enhancing tool used to organize information related to applicants for temporary residence, undermines the reasonableness of the Officer’s decision.
….
B. Procedural Fairness
(1) Use of Chinook Processing Tool
[28] The Applicant submits that the Officer’s use of the Chinook processing tool to assist in the assessment of the application is procedurally unfair. The Applicant contends that the tool, which he claims is able to extract information from the GCMS for many applications at a time and generate notes about these applications in
“a fraction of the time”it would take to review an application otherwise, results in a lack of adequate assessment of the Applicant’s work permit application.[29] The Respondent submits that IRCC’s use of the Chinook tool to improve efficiency in addressing a voluminous number of temporary residence applications does not amount to a specific failure of procedural fairness in the Applicant’s case. The Respondent notes that the Applicant has failed to point to any evidence to support that the Officer’s use of the Chinook tool resulted in the omission of a key consideration in the assessment of his application or deprived him of the right to have his case heard. The Respondent contends that the Applicant’s submissions appear to be little more than an objection to IRCC’s use of this tool.
[30] I agree with the Respondent. While it was open to the Applicant to raise the ways that the Chinook processing tool specifically resulted in a breach of procedural fairness in the Officer’s assessment of his case, he has not provided any evidence of such a connection. I would also note that the Chinook tool is not intended to process, assess evidence, or make decisions on applications, and the Applicant has failed to raise any evidence countering this or demonstrating that the tool impacts the fairness of the decision-making process. (emphasis added)
Note – again there appears to be a lack of evidence filed. However I do take issues with the “Chinook tool is not intended to process, assess evidence’ portion. I think there is not enough on the record or in what IRCC has publicly shared to make that statement. It is a processing tool at the end of the day, so a processing tool does process and based on what we know about the modules work (especially Module 5’s risk indicators and local word flags) that it definitely assesses and ‘flags’ the evidence at the very least.
[4]Haghshenas v. Canada (Citizenship and Immigration), 2023 FC 464 (CanLII), <https://canlii.ca/t/jwhkd> – Justice Brown, Dismissed
[24] As to artificial intelligence, the Applicant submits the Decision is based on artificial intelligence generated by Microsoft in the form of
“Chinook”software. However, the evidence is that the Decision was made by a Visa Officer and not by software. I agree the Decision had input assembled by artificial intelligence, but it seems to me the Court on judicial review is to look at the record and the Decision and determine its reasonableness in accordance with Vavilov. Whether a decision is reasonable or unreasonable will determine if it is upheld or set aside, whether or not artificial intelligence was used. To hold otherwise would elevate process over substance. (emphasis added)….
[28] Regarding the use of the
“Chinook”software, the Applicant suggests that there are questions about its reliability and efficacy. In this way, the Applicant suggests that a decision rendered using Chinook cannot be termed reasonable until it is elaborated to all stakeholders how machine learning has replaced human input and how it affects application outcomes. I have already dealt with this argument under procedural fairness, and found the use of artificial intelligence is irrelevant given that (a) an Officer made the Decision in question, and that (b) judicial review deals with the procedural fairness and or reasonableness of the Decision as required by Vavilov. (emphasis added)
What arises from the above is precious court resources were spent on four identical cases from the same counsel, making identical arguments, rendering nearly identical judgments all on summating ‘we do not have enough in front of us.’
One wonders if the Department of Justice should have just heeded Justice Little’s comments in Ocran v. Canada (Citizenship and Immigration), 2022 FC 175 (CanLII), <https://canlii.ca/t/jmk0l> to ask for a reference but as these tools are constantly evolving, I do understand the trepidation and costs:
V. Matters Raised by the Respondent
[57] The respondent raised additional matters for resolution by this Court about the preparation of GCMS notes generally by visa officers using spreadsheets made with a software-based tool known as the
“Chinook Tool”. The respondent sought to resolve an issue about whether contents of Certified Tribunal Record (“CTRs”) were deficient because the spreadsheets are not retained and therefore do not appear in the CTRs prepared for matters such as this application. The respondent also purported to file an affidavit in an effort to provide a factual foundation; the applicant objected to its admissibility and relevance to the proceeding.[58] In my view, the Court should not resolve the additional matters raised by the respondent on this application. There is no dispute or controversy between these parties about the contents of the CTR or the officer’s preparation of the GCMS notes. The respondent’s written submissions acknowledged that the additional matters were unrelated to the substantive merits of this application for judicial review and that the applicant did not raise any issues related to the CTR, including whether it was deficient. The respondent did not file a Notice of Application and did not provide a legal basis for the Court to adjudicate the matters on an advisory basis. In the circumstances, the respondent’s issues should be resolved in another case in which there is a live dispute on the facts. (emphasis added)
Why is it important to counsel to fight Chinook?
Chinook has been used to bulk process, specifically categorically bulk approve and refuse applicants from high volume visa offices – specifically in the Global South. The Federal Court first learned about the use of Chinook in Ocran, although another case (eventually abandonned) was chosen as the initial pilot. IRCC developed Chinook in 2018, but largely build it out during the COVID-19 pandemic which allowed them to continue to process applications at such a high clip.
Most of the public knowledge about Chinook has come from four documents
- Andie Daponte’s affidavit in Ocran;
- Andie Daponte’s cross-examination transcript in Ocran;
- CIMM — Chinook Development and Implementation in Decision-Making – February 15 & 17, 2022
- CIMM Report of the Standing Committee on Citizenship and Immigration – “Differential Treatment in Recruitment and Acceptance Rate of Foreign Students in Quebec and in the Rest of Canada” (see especially pages 66-69)
Interestingly enough of the decision in the tetralogy actually cited to the third document, but I have yet to see any cited to the fourth.
Finally, what is missing is the many Chinook manuals (largely redacted) that contain Officer guidance on how to utilize the tool. Much of this guidance, speaking generously, puts into question some of the information in Document 1 and 2 above – see below screenshots for just a few examples.
There have been two cases, both Justice Grammond decisions, that have raised some concerns about how Chinook may contribute to boilerplate language in refusals.
Safarian v. Canada (Citizenship and Immigration), 2023 FC 775 (CanLII), <https://canlii.ca/t/jxm61>
[3] In the present case, the officer’s notes consist largely of boilerplate statements that we see repeatedly in study permit decisions and that appear to be generated by the Chinook software. As I explained in Boukhanfra v Canada (Citizenship and Immigration), 2019 FC 4 at paragraph 9, the use of boilerplate is not in itself objectionable, but the reviewing court must be satisfied that the decision-maker turned their minds to the facts of the case. The fact that the use of a particular sentence was held to be reasonable in a previous case does not immunize it from review in subsequent cases. Conversely, a sentence found to be unreasonable in a particular context will not necessarily be so in a different context. In the end, the court must be able to understand why the decision-maker reached a particular conclusion. (emphasis added)
Khosravi v. Canada (Citizenship and Immigration), 2023 FC 805 (CanLII), <https://canlii.ca/t/jxn8b>
[12] I note that Ms. Khosravi’s application was
“processed with the assistance of Chinook 3+”. I do not know if the shortcomings outlined above result from the use of this tool. I will simply say that the use of assisted decision-making tools does not relieve officers from the duty to fully consider an application, most importantly the study plan. If the use of such a tool gives the officer a truncated vision of the application, the resulting decision may well be unreasonable. (emphasis added)
I think Justice Grammond’s comments that tools that truncate an application and spit out boilerplate refusal language, , although likely in obiter, is precisely the ‘door opener’ to concerns we have been discussing about Chinook. Again, however, it appears in neither case there was an evidentary required on Chinook.
It is important to note at this stage, that we are still awaiting the release of the much anticipated Gender Based Analysis (GBA+) Report on Chinook, we have learned that the use of Chinook has correlated with higher refusal rates for study permits, but causation likely cannot be established due to other factors such as the pandemic and political shifts during this time. In all four of the tetralogy cases, it was the applicant NOT the Department of Justice that raised Chinook as an issue.
Chinook, as I have discussed here, is actively being relaced by Cumulus, including the AAM module equivalent to Module 1 in Chinook. As far as I currently know, Cumulus is only being used in processing family class/spousal sponsorships but will be used eventually for temporary residents in a very similar way to the current Chinook tool. Indeed, the main different appears to be both the enhanced cloud/privacy functions but also the integration of Cumulus with GCMS.
Could the FC Have Avoided the Tetralogy?
Again, the timing likely was not fortuitious in a case like this but I would suggest there are probably some practical technological solutions that could have been implemented.
a) Applicant’s Record Duplication Checker
While I am not suggesting that the Federal Court necessarily should be adopting TurnItIn (I just gave a bunch of former students grief with that word, I apologize) – surely there’s a way that documents that are already OCR’d can be checked for duplication and replication. Whether it is the same counsel and copy and pasting memos without adjusting the facts OR it is newer counsel to the space utilizing the Federal Court’s open court policy to get arguments and just copy and paste replicating, this has the potential of being problematic. A technology that flagged that this was the same case x 4, at various stages could likely have at least flagged this prior to hearing for the Judges.
b) FC Ordering Identical Cases Be Heard Together
While this is usually the Applicant’s strategic play when trying to run more affordable group litigation, perhaps this is an instance where the Court should have stepped in and said – there are four similar cases, we want it to be case managed together subject to any submissions to the otherwise. Perhaps this is a good case to switch the onus onto applicants (I am probably getting flamed by fellow applicant’s counsel for this one)
c) Requesting Written Submissions from the Parties Specifically on Chinook
One step that I have seen Federal Court judges do, although in most cases this occurs when new case law or issues arise, prior to hearing and before decisions are made is to request written submission from the parties on an emerging topic.
Perhaps, doing this in the first case to see if counsel was actually prepared to raies the concerns could have avoided it getting to the point of being called baseless musings.
I would have to look into the Federal Court Immigration Rules a bit more but at this stage could the Court simply dispose of the matter in writing.
d) Dismissing Leave With Reasons or Adding a Mechanism Pre-Leave to Express Concern
This seems like the obvious answer, but several judges rendered positive leave decisions on the same factual record and argument. Sure there is some discrepancy between arguable case and actually obtaining one’s result at the end of the day, but if arguments appeared to fall so fundamentally short on what ended up being the core argument of the applicants, is there not a pre-leave recourse.
Again, this one would probably take the most work as currently there are no reasons associated with Leave Decisions. I am personally in the camp that even one or two sentences pointing out why the threshold for leave was met/not met would be a welcome step to greater judicial transparency.
At the end of the day (and at the end of the bill), the Court has heard four times, but it should have likely only heard once. Legal fees were incurred for four different cases, four different judges had to expend energy, and the state of the law is no clearer.
What is clear is applicants have to do better when arguing Chinook and actually build a factual foundation, utilizing the affidavit process and other means of challenging, motioning, and cross-examining, to get there.
Do you have any creative solutions to avoid the Court going through the tetralogy and duplicating resources? Would love to discuss further.
