GCMS

Award-Winning Canadian Immigration and Refugee Law and Commentary Blog

Blog Posts

Predictive/Advanced Analytics + Chinook – Oversight = ?

In September 2021’s issue of Lexbase, my mentor Richard Kurland, provides further insight into what happens behind the scenes of Immigration, Refugees, and Citizenship Canada (“IRCC”) processing, specifically providing a section titled: “Overview of the Analytics-Based Triage of Temporary Resident Visa Applications.

At the outset, a big thank you to the “Insider” Richard Kurland for the hard digging that allows for us to provide this further analysis.

 

What the Data Suggests

I encourage all of you to check out the first two pages from the Lexbase issue, as it contains direct disclosure from IRCC’s Assistant Director, Admissibility opening up the process by way Artificial Intelligence is implemented for Temporary Resident Visas (‘TRVs’), specifically in China and India, the two countries that have implemented it so far. By way of this June 2020 disclosure, we confirm that IRCC has been utilizing these systems for online applications since April 2018 for China, August 2018 for India, and for Visa Application Centre (“VAC”) based applications since January 2020.

To summarize (again – go read Lexbase and contact Richard Kurland for all the specific details and helpful tables), we learn that there is a three Tier processing system in play. This filters the simplest applications (Tier 1), medium complexity applications (Tier 2), and higher complexity applications (Tier 3). While human officers are involved in all three Tiers, Tier 1 allows a model to recommend approval based on analytics, where as Tier 2 and Tier 3 are flagged for manual processing. IRCC claims that the process is only partially automated.

The interesting factor, and given we have been as a law firm focusing a lot on India, is how the designated of a Tier 2 file drives the approval rates from the high nineties (%) to 63% for online India apps to 37%  for India VAC applications. Moving to Tier 3, it is only 13% for online India and 5% for India VAC. The deeming of a file Tier 3 appears to make refusal a near surety.

What is fascinating is how this information blends usage of “Officer Rules,” the first stage filter which  actually precedes the computerized Three Tier triages and is targeted at cases with higher likelihood of ineligibility or inadmissibility.

The Officer Rules system would be the system utilized at other global visa offices that do not use the computerized AI decision-making of India and China. Looking specifically at the case of India, the Officer Rules system actually approves cases at a much higher rate (53% for online India, and 38% for India VAC).

These rates are in-fact comparable to Tier 2 moderately complex cases – ones that presumably do not contain the serious ineligibility and inadmissibility concerns of Officer Rules or Tier 3 . It suggests that the addition of technology can sway even a moderately complex case into the same outcomes as a hand-pulled out complex case.

Ultimately, this suggests that complete human discretion or time spent assessing factors can be much more favourable than when machines contribute to overall decision-making.

It Comes Down to Oversight and How These Systems Converge

Recently, we’ve been discussing in Youtube videos (here and here), podcasts, and articles about IRCC’s Chinook system for processing applications. Using an excel-based model (although moving now to an Amazon-based model in their latest version), applicants data are extracted into rows, that contain batch information for several applicants, presumably allowing for all the analytics to be assessed.

Given we know IRCC takes historic approval rates and data as a main driving factor, it is reasonable to think Immigration Officers are given these numbers as internal targets. I am sure, as well, that with major events like COVID and the general dissuasion of travel to Canada, that these goalposts can be moved and expanded at direction.

An excel-based system tracking approvals and refusals likely put these stats front and centre to an officer’s discretion (or a machine’s) on an application. Again to utilize a teaching analogy (clearly I miss teaching), I utilized a similar ‘Speedgrader’ type app which forced me, mid-marking, to often to revisit exams that I had already graded because I had awarded the class average marks that were too high. I have no doubt a parallel system exists with IRCC.

What this all means, as my colleague, Zeynab Ziaie has pointed out in our discussions, there are major concerns that Chinook and the AI systems have not been developed and rolled out with adequate lawyer/legal input and oversight, which leads to questions about accountability. Utilizing the Chinook example, what if the working notes that are deleted contain the very information needed to justify or shed light on how an application was processed.

My question, in follow-up, is how are the predictive/advanced analytics systems utilized by India and China for TRVs influencing Chinook? Where is the notation to know whether one’s file was pre-assessed by “Officer’s Rule” or through the Tiers. I quickly reviewed a few GCMS notes prior to this call, and though we know whether a file was pre-accessed, we have no clue which Tier it landed on.

Furthermore, how do we ensure that the visa-office subjective “Officer Rules” or the analytical factors that make up the AI system are not being applied in a discriminatory manner to filter cases into a more complex/complex stream. For example, back in 2016 I pointed how the Visa-Office training guides in China regionally and geographically discriminate against those applying from certain Provinces assigning character traits and misrepresentation risks. We know in India, thanks to the work of my mentor Raj Sharma, that the Indian visa offices have a training guide on genuine relationships and marriage fraud that may not accord with realities.

Assuming that this AI processing system is still being used only for TRVs and not for any other permits, it must be catching (with the assistance of Chinook’s key word indicators no less) words such as marriage, the names of rural communities, marital status, perhaps the addresses of unauthorized agents, and businesses that often have been used as a cover for support letters. Within that list there’s a mix of good local knowledge, but also the very stereotypes that have historically kept families apart and individuals from being able to visit without holding a study permit or work permit.

If we find out, for example, that filtering for complex cases only happens at visa offices with high refusal rates or in the Global South, does that make the system unduly discriminatory?

We acknowledge of course that the very process of having to apply to enter the borders, the division of TRV and electronic Travel Authorization (eTA) requiring countries is discriminatory by nature, but what happens when outcomes on similar facts are so discrepant?

In other areas of national bureaucracy, Governments have moved to blind processing to try and limit discrimination around ethnic names, or base decisions on certain privileges (ability to travel and engage in previous work), and remove identifying features that might lead to bias. For immigration it is the opposite, you see their picture, their age, and where they are from, and why they want to come (purpose of visit). As we have learned from Chinook, that is the baseline information that is being extracted for Officers to base their decisions on.

When – as a society – do we decide to move away (as we have) on what were once harmful norms to new realities? Who is it that makes the call or calls for reviews for things such as consistency or whether a particular discriminatory input in the AI system is no-longer consistent with Charter values?

Right now, it is all in the Officer’s discretion and by extension, the Visa Offices, but I would recommend some unified committee of legal experts and race/equity scholars need to be advising on the strings of the future, inevitable, AI systems. This would also unify things across visa offices so that there is less discrepancy in the way systems render decisions. While it makes sense that heavier volume visa offices have more tools as their disposal, it should not depend on where you live to receive less access to human decision-makers or to an equal standard of decision-making. We do not want to get to a place where immigration applicants are afraid to present their stories or speak their truths for fear of being filtered by artificial intelligence. From my perspective, we are better of being transparent and setting legitimate expectations.

What are your thoughts on the introduction of AI, the interaction with Chinook, and the need for oversight? Feel free to engage in the comments below or on social media!

Thanks again for reading.

Read More »

Three Things You Likely Don’t But Should Know About How IRCC Assesses Your Study Permit Application

#1 – Your Application is Decided Using a Microsoft Excel Spreadsheet

It might come as a surprise to you that IRCC utilizes the classic, but with some tech additions, software of Microsoft Excel to decide temporary resident applications.

The Officer essentially provides all the information about a client in a row with several columns (including working notes – more on this later). This allows them to process multiple applications utilizing one screen. There are also multiple clients that make up the constituent rows.

Each column within a row contains information regarding the Applicant’s name, age, purpose of visit, date of receipt of the application, citizenship of the client, and previous travel. It appears that some of this information is pre-assessed by a processing Officer, but much of it comes directly from the IMM forms.

You can see here how the pre-assessment notes show up with respect to the ‘verbose client information.’ Verbose client information appears to be information directly from the forms. This suggests that the pre-assessment plans a significant and important role in an application. While it is seemingly ‘blind’ one can see from the below that if you break it down to age, gender, marital status, and citizenship that many of the personal identifying features that can make a young, single, and mobile woman applicant from Zimbabwe difficult to make to record. The Pre-assessment also shows (based on categories selected) that previous travel and proof of funds continue to be important factors. As such, it is difficult to say if travel history is as ‘neutral’ as the Federal Courts have attempted to establish it is.

Why do they do it this way via excel? Well IRCC claims that officers can increase their processing volume (depending on visa office) anywhere from 5% to 35% using this system.

I would also not be surprised (I am speculating) if the excel-based system allowed also for real-time tracking of statistics. This way a visa office with a refusal target could likely keep track while at the same time processing applications.

 

#2 – Reasons are Templated and Generated After Refusal. They Don’t Have to Refer to Your Original Evidence (their position, not mine)

If one were to think of it logically, or perhaps engage in the exercise themselves, it would make sense to do some sort of a yay/nay list on a chart or table and ultimately decide, based on the facts gathered, whether or not to approve a client. Indeed, while not required, much of immigration (think Ribic factors or the assessment of humanitarian and compassionate grounds) often work on this weighing system.

Such is not necessarily the case with temporary resident refusals. With IRCC’s systems, a decision to refuse or approve is made first, and then a notes generator (read: template generator) is utilized to choose the applicable reasons. The Officer then copies and pastes this into Global Case Management System (GCMS).

While Officers continue to have access to the original documents submitted by applicants, much of the guidance suggests the anchoring point is the excel document – one populated by the aforementioned pre-assessed notes and verbose client information. Officers are very much deciding to approve or refuse simply on an individual’s basic profile. This suggests that whatever is chosen to be extracted from an application, rather than what is actually in the application is most important. Such guidance should serve as a reminder to keep support letters and evidence not only strong, but visible and searchable rather than tucked away on page 12 of a 15 page submission letter.

There are also ‘risk indicators’ and ‘local word flags.’ Risk indicators can capture where there is a trend, for example, of an Office submitting fraudulent information and local word flags, capture words such as ‘wedding.’ I am still researching what the other words are, but we know they depend on what visa office runs them. It would not be a surprise to see more risk indicators and local  I would not be surprised if IRCC is also running OCRs (optical character recognition) or utilizing machine-based decision-makers to flag key words. Yet, looking at the GCMS notes of several recent files, it appears risk indicators and local word flags don’t often appear. What this may suggest, is that the Officers rely more on the pre-assessment, verbose info, and their working notes to render a decision.

Which brings us to the issue of working notes below.

 

#3 Working Notes of Officers (i.e. Where the Factual Analysis Takes Place) are Not Ordinarily Retained

Templated reasons themselves do not (at this stage) need to contain reference to facts in the Application. While IRCC maintains that Officers do have the right to choose not to use them, the reality is any officer facing instructions to process fast and maintain consistency, likely won’t diverge too far from them.

When clients come and find me after a referral, I often hear from them that they believe the Officer ignored evidence or turned a blind eye to something they submitted. Turns out there is likely much more to it.

Officers do have space to maintain working notes in their system, but – and importantly, these notes are not transferred to GCMS for privacy and administrative convenience purposes. IRCC claims that if they were required to manually input Officer’s working notes it would create too much of an admin burden.

Strategically though, this is a brilliant play. If decisions were to include working notes and commentary it would open up the possibility of all sorts of litigation. Thinking back in history, it was the working notes of several Officers that led to such a departmental disaster such as Baker. 

The Supreme Court of Canada’s decision in Vavilov also supports short, pithy reasons that maintain consistency – essentially what IRCC is trying to do with this system.

The Majority writes at paragraph 77:

[77]                          It is well established that, as a matter of procedural fairness, reasons are not required for all administrative decisions. The duty of procedural fairness in administrative law is “eminently variable”, inherently flexible and context-specific: Knight v. Indian Head School Division No. 19, [1990] 1 S.C.R. 653, at p. 682; Baker v. Canada (Minister of Citizenship and Immigration), [1999] 2 S.C.R. 817, at paras. 22-23; Moreau-Bérubé, at paras. 74‑75; Dunsmuir, at para. 79. Where a particular administrative decision-making context gives rise to a duty of procedural fairness, the specific procedural requirements that the duty imposes are determined with reference to all of the circumstances: Baker, at para. 21. In Baker, this Court set out a non-exhaustive list of factors that inform the content of the duty of procedural fairness in a particular case, one aspect of which is whether written reasons are required. Those factors include: (1) the nature of the decision being made and the process followed in making it; (2) the nature of the statutory scheme; (3) the importance of the decision to the individual or individuals affected; (4) the legitimate expectations of the person challenging the decision; and (5) the choices of procedure made by the administrative decision maker itself: Baker, at paras. 23-27; see also Congrégation des témoins de Jéhovah de St-Jérôme-Lafontaine v. Lafontaine (Village), 2004 SCC 48, [2004] 2 S.C.R. 650, at para. 5. Cases in which written reasons tend to be required include those in which the decision-making process gives the parties participatory rights, an adverse decision would have a significant impact on an individual or there is a right of appeal: Baker, at para. 43; D. J. M. Brown and the Hon. J. M. Evans, with the assistance of D. Fairlie, Judicial Review of Administrative Action in Canada (loose-leaf), vol. 3, at p. 12-54.

(emphasis added)

IRCC is in the process (the part I am not yet at the liberty to discuss as I am assisting in the litigation) of getting judicial endorsement for their choice of process. I can say that if they are successful, and given it is well-established that the impact to foreign nationals (such as students is on the low end), this could serve as a rubber stamp. Such a process would make future judicial reviews much more difficult if the Courts find that templated reasons do not need factual reference. Furthermore, refusal letters could simply that the Applicant’s evidence was insufficient and leave it to their counsel, Department of Justice, to build up a justification after the fact.

 

Caution: Expect GCMS Notes to Thin at the Detriment to Your Client’s Knowledge of the Case to Be Met

What does this all practically mean for your run of the mill temporary resident applicant. Well – expect GCMS notes to say less and less and for the bulk of the information to be retained on IRCC’s ‘internal’ system. It is also likely moving forward (and that we’ve already seen in some cases) nothing in the notes for when procedural fairness letters are sent. This will make it very difficult to respond, especially where procedural fairness letters are so broadly worded. This could make the process much less transparent and lead to many more misrepresentation finding (as just one example).

 

Bonus: A Little Gratitude

I want to thank again, my incredible colleague Zeynab Ziaie for her advocacy and supporting our efforts to learn more about the way IRCC operates. I have purposely not included anything in this piece that may be subject of our litigation and is not already publicly accessible. We may be writing more about this shortly.

Read More »
About Us
Will Tao is an Award-Winning Canadian Immigration and Refugee Lawyer, Writer, and Policy Advisor based in Vancouver. Vancouver Immigration Blog is a public legal resource and social commentary.

Let’s Get in Touch

Translate »