ATIP

Award-Winning Canadian Immigration and Refugee Law and Commentary Blog

Blog Posts

Predictive/Advanced Analytics + Chinook – Oversight = ?

In September 2021’s issue of Lexbase, my mentor Richard Kurland, provides further insight into what happens behind the scenes of Immigration, Refugees, and Citizenship Canada (“IRCC”) processing, specifically providing a section titled: “Overview of the Analytics-Based Triage of Temporary Resident Visa Applications.

At the outset, a big thank you to the “Insider” Richard Kurland for the hard digging that allows for us to provide this further analysis.

 

What the Data Suggests

I encourage all of you to check out the first two pages from the Lexbase issue, as it contains direct disclosure from IRCC’s Assistant Director, Admissibility opening up the process by way Artificial Intelligence is implemented for Temporary Resident Visas (‘TRVs’), specifically in China and India, the two countries that have implemented it so far. By way of this June 2020 disclosure, we confirm that IRCC has been utilizing these systems for online applications since April 2018 for China, August 2018 for India, and for Visa Application Centre (“VAC”) based applications since January 2020.

To summarize (again – go read Lexbase and contact Richard Kurland for all the specific details and helpful tables), we learn that there is a three Tier processing system in play. This filters the simplest applications (Tier 1), medium complexity applications (Tier 2), and higher complexity applications (Tier 3). While human officers are involved in all three Tiers, Tier 1 allows a model to recommend approval based on analytics, where as Tier 2 and Tier 3 are flagged for manual processing. IRCC claims that the process is only partially automated.

The interesting factor, and given we have been as a law firm focusing a lot on India, is how the designated of a Tier 2 file drives the approval rates from the high nineties (%) to 63% for online India apps to 37%  for India VAC applications. Moving to Tier 3, it is only 13% for online India and 5% for India VAC. The deeming of a file Tier 3 appears to make refusal a near surety.

What is fascinating is how this information blends usage of “Officer Rules,” the first stage filter which  actually precedes the computerized Three Tier triages and is targeted at cases with higher likelihood of ineligibility or inadmissibility.

The Officer Rules system would be the system utilized at other global visa offices that do not use the computerized AI decision-making of India and China. Looking specifically at the case of India, the Officer Rules system actually approves cases at a much higher rate (53% for online India, and 38% for India VAC).

These rates are in-fact comparable to Tier 2 moderately complex cases – ones that presumably do not contain the serious ineligibility and inadmissibility concerns of Officer Rules or Tier 3 . It suggests that the addition of technology can sway even a moderately complex case into the same outcomes as a hand-pulled out complex case.

Ultimately, this suggests that complete human discretion or time spent assessing factors can be much more favourable than when machines contribute to overall decision-making.

It Comes Down to Oversight and How These Systems Converge

Recently, we’ve been discussing in Youtube videos (here and here), podcasts, and articles about IRCC’s Chinook system for processing applications. Using an excel-based model (although moving now to an Amazon-based model in their latest version), applicants data are extracted into rows, that contain batch information for several applicants, presumably allowing for all the analytics to be assessed.

Given we know IRCC takes historic approval rates and data as a main driving factor, it is reasonable to think Immigration Officers are given these numbers as internal targets. I am sure, as well, that with major events like COVID and the general dissuasion of travel to Canada, that these goalposts can be moved and expanded at direction.

An excel-based system tracking approvals and refusals likely put these stats front and centre to an officer’s discretion (or a machine’s) on an application. Again to utilize a teaching analogy (clearly I miss teaching), I utilized a similar ‘Speedgrader’ type app which forced me, mid-marking, to often to revisit exams that I had already graded because I had awarded the class average marks that were too high. I have no doubt a parallel system exists with IRCC.

What this all means, as my colleague, Zeynab Ziaie has pointed out in our discussions, there are major concerns that Chinook and the AI systems have not been developed and rolled out with adequate lawyer/legal input and oversight, which leads to questions about accountability. Utilizing the Chinook example, what if the working notes that are deleted contain the very information needed to justify or shed light on how an application was processed.

My question, in follow-up, is how are the predictive/advanced analytics systems utilized by India and China for TRVs influencing Chinook? Where is the notation to know whether one’s file was pre-assessed by “Officer’s Rule” or through the Tiers. I quickly reviewed a few GCMS notes prior to this call, and though we know whether a file was pre-accessed, we have no clue which Tier it landed on.

Furthermore, how do we ensure that the visa-office subjective “Officer Rules” or the analytical factors that make up the AI system are not being applied in a discriminatory manner to filter cases into a more complex/complex stream. For example, back in 2016 I pointed how the Visa-Office training guides in China regionally and geographically discriminate against those applying from certain Provinces assigning character traits and misrepresentation risks. We know in India, thanks to the work of my mentor Raj Sharma, that the Indian visa offices have a training guide on genuine relationships and marriage fraud that may not accord with realities.

Assuming that this AI processing system is still being used only for TRVs and not for any other permits, it must be catching (with the assistance of Chinook’s key word indicators no less) words such as marriage, the names of rural communities, marital status, perhaps the addresses of unauthorized agents, and businesses that often have been used as a cover for support letters. Within that list there’s a mix of good local knowledge, but also the very stereotypes that have historically kept families apart and individuals from being able to visit without holding a study permit or work permit.

If we find out, for example, that filtering for complex cases only happens at visa offices with high refusal rates or in the Global South, does that make the system unduly discriminatory?

We acknowledge of course that the very process of having to apply to enter the borders, the division of TRV and electronic Travel Authorization (eTA) requiring countries is discriminatory by nature, but what happens when outcomes on similar facts are so discrepant?

In other areas of national bureaucracy, Governments have moved to blind processing to try and limit discrimination around ethnic names, or base decisions on certain privileges (ability to travel and engage in previous work), and remove identifying features that might lead to bias. For immigration it is the opposite, you see their picture, their age, and where they are from, and why they want to come (purpose of visit). As we have learned from Chinook, that is the baseline information that is being extracted for Officers to base their decisions on.

When – as a society – do we decide to move away (as we have) on what were once harmful norms to new realities? Who is it that makes the call or calls for reviews for things such as consistency or whether a particular discriminatory input in the AI system is no-longer consistent with Charter values?

Right now, it is all in the Officer’s discretion and by extension, the Visa Offices, but I would recommend some unified committee of legal experts and race/equity scholars need to be advising on the strings of the future, inevitable, AI systems. This would also unify things across visa offices so that there is less discrepancy in the way systems render decisions. While it makes sense that heavier volume visa offices have more tools as their disposal, it should not depend on where you live to receive less access to human decision-makers or to an equal standard of decision-making. We do not want to get to a place where immigration applicants are afraid to present their stories or speak their truths for fear of being filtered by artificial intelligence. From my perspective, we are better of being transparent and setting legitimate expectations.

What are your thoughts on the introduction of AI, the interaction with Chinook, and the need for oversight? Feel free to engage in the comments below or on social media!

Thanks again for reading.

Read More »

Regionalism and Geographic Discrimination in Canadian Immigration? – A China Case Study

Recently, my colleague Steven Meurrens uncovered via Access to Information and Privacy Request officer training guides used by Immigration, Refugee and Citizenship Canada (“IRCC”) Beijing’s Temporary Resident Unit from 2013-2015.

These guides are enlightening, opening our eyes to the inner workings of an overseas visa office. While I won’t share the entire ATIP here, the pages range from discussion on officer working hours, to how to organize files, and importantly how to prescreen and assess temporary resident applicants.

Steve broke the scoop by tweeting the following:

I followed with another tweet, disclosing a few more pages providing greater insight into the way Chinese applicants were viewed by region:

Yesterday, I was interviewed by Ming Pao Newspaper to share my views on the content. The Ming Pao piece (in Mandarin)  has so far stirred a lot of reaction within the Chinese community as individuals debate whether IRCC’s practices are fair in this regard.

I want to take the context of this situation outside of just the Chinese community a little, and share a few more tidbits and why I think this training guide opens up a vulnerability in IRCC’s decision-making process that may face judicial challenges moving forward.

Whether or not files are re-opened because of the revelation that regional discrimination is taking place is  beyond the scope of this post, However, I certainly want to also express my position that regionalism, regional or geographic discrimination, is likely not a reasonable basis for refusal that can be relied on by Canadian immigration officers moving forward. However, at the same time, I question whether it is inevitable given the already inherent biases of our Temporary Resident Visa regime.

What We Learned From the ATIP Results

IMG_1729

The above map is very important and telling. As I mentioned in my initial tweet, this map is from a training guide titled 2015 while the one Steve posted earlier  was from 2013- 2014. The 2014 one is important for the line that states “40% of claimants (I am assuming this means IRCC applicants) were from the provinces Liaoning, Guangdong, Hebei, Fujian, and Henan. It states further that “borderline” cases should be refused.

The 2015 map is revealing in that it appears to add even more specific detail. In addition to the four above it adds Tianjin (a special economic zone in Northern China) and the province of Hunan. Because of the black and white nature of the ATIP results we cannot see the entire colour scale of the map which in addition to the circled “risk provinces” appears to show a sliding scale of risk.

The side note states that the risk is in regard to “genuineness” which taking it a step further can equate to risk of misrepresentation. Misrepresentation as we know caries with in a 5-year bar and exclusion order and is a finding that often affects Chinese applicants who often times are poorly represented by incompetent counsel.

Overall, as per this chart, these are the provinces where applicants are presumed to have a greater likelihood to be misrepresenting on their applications.

We receive even further insight from the chart below which set out numerically what the risks are.

IMG_1735

What is fascinating in the above chart is that different cities within the provinces across China are giving numbers reflecting the risk. Again, we don’t have access to the  exact scale or the process by which the numbers are assigned.

However, extrapolating the fact that Beijing and Suzhou receives high numbers and the cities that are/are not redacted, we can assume the higher the number, the lower the risk. The redacted portions therefore reflect therefore the riskiest regions as aligning with the “group of seven” high risk regions.

We also can deduce through this chart what each of the colours and categories represent.

IMG_1731

While the above is unfortunately redacted for colour, we see here on another page how the colour coding system works.

IMG_1733

Breaking down the categories further

  • Green = Visiting families or Friends
  • Pink = Tourist (includes large approved ADS/ADS-P Groups)
  • Green = Visiting Family or Friends
  • Grey = Business – Private Delegations
  • Orange = Business – Personal or Professional Affairs;
  • Krat/Tan = Transit
  • Blue – PRTD = previous landed – require a Travel Document
  • Purple: Short Term (<5 months) students.

Below the labels clarify further that

  • CAN + are individuals with a U.S visa or Previous travel to Canada/US in past 5-10 years.

Here we can assume the increased risk label in  the chart is put on perceived “tourists” from Tianjin, Hebei, Liaoning, Jilin etc and “private delegations/business” from same.

Notwithstanding  the above, the training materials themselves clarify that Beijing’s approval rate is 87% – giving the perception that this is to be a standard to be met.

 

 

Where is the “Genuineness” and “Risk Evidence” coming from?

The first question that pops up is where the evidentiary burden behind this. As IRCC trains their officers and provides them a global map and provinces that they should look out for- what would the answer be if a trainee officer asked – “why Hunan over Yunan?”

Does IRCC have in its coffers a master statistics list of applicants and their hometowns, number of refusal per hometown, and results on appeal?

In the alternative, could the entire basis for the risk factors be on the ground anecdotal data.

What is striking about the list for China is that the regions that are highlighted as being at risk for genuineness are also among the provinces that receive the most regional discrimination from Chinese within China.

For example. the prejudice felt by individual from the central provinces of Henan and Hebei have been covered frequently in Chinese media. See here for example and Dan Harris’s China Law Blog post on regional employment discrimination here. 

Is the Chinese media potentially feeding into these perceptions? I don’t find it ironic that within China, I also hear from individual’s on the street similar regional biases to the “dongbeiren” of the North for being more aggressive and harder to deal with, or the “central chinese” as being farmers, or the Cantonese being “shady.”  How much are these stereotypes (purely prejudicial comments with no factual  basis) seeping into Chinese policy?

Presence of  Global Risk Tiering Chart

The Global Risk Tiering chart is fascinating. Each city appears to have a country code and the codes appear to be hundreds of numbers off assuming there are more than just the  list here. The ORG/Entity classification code appears to mirror almost a Designated Learning Institute Code.

If I were to select Jalandhar, Manila, or Tehran would I get results? Would there be a similar number system?

Are these practice utilized by Beijing in their visa office unique to the visa office or shared across different visa offices? Particularly, in this day and age where applicants primarily apply online and IRCC has the ability to process in Canada or send abroad to any visa office,  what roles do the risk tiering chart play (potentially in place of on the ground knowledge)?

What if I am from a “genuineness” risk Province and my application is refused?

If these cases where positive and negative temporary residence factors both exist and the difference maker is based on the province the applicant is from, I would argue procedural fairness and reasonableness issues both may be triggered in challenging refusals.

From a procedural fairness perspective, I would argue that this risk tiering chart would be considered extrinsic evidence not available to the Applicant. It could not be reasonably expected that the Applicant is aware that their region of residence is particularly at risk unless the charts were made public (which would have never occurred but for this ATIP). A lack of genuineness if equated to a “lack of credibility” may also trigger the requirement for concerns to be put directly to the Applicant rather than immediately refusing as the training guide suggests. If refused, the Applicant could argue that they were not given a reasonable opportunity to respond to the credibility issues.

From a reasonableness standard, as highlighted by the recent decision of the Supreme Court of Canada in Kanthasamy, it is not reasonable, and a fettering of discretion, to treat IRCC policy as a legal standard.

So far, issues such as travel history (which cannot be a stand-alone reason for refusal) have been tied into the issue of ties to the home country as a permissible contributor to refusals. I would suggest that regionalism as a factor for refusal begins to create major challenges for IRCC as it would be difficult to link the fact an applicant is from a certain part of a country as evidence that they would not leave Canada or would not comply with the terms of their stay in Canada. I think fettering discretion is a serious consideration in these cases.

Conclusion

As I mentioned in the Ming Pao piece, I see this ATIP as the tip of the iceberg. I have a feeling that there may be grounds for opening several files to look at how the temporary resident factors were balanced in the decision.

Remember the law states:

Obligation on entry
  • (1) Every foreign national, other than a foreign national referred to in section 19, who seeks to enter or remain in Canada must establish,

    • (a) to become a permanent resident, that they hold the visa or other document required under the regulations and have come to Canada in order to establish permanent residence; and

    • (b) to become a temporary resident, that they hold the visa or other document required under the regulations and will leave […]

Read More »
About Us
Will Tao is an Award-Winning Canadian Immigration and Refugee Lawyer, Writer, and Policy Advisor based in Vancouver. Vancouver Immigration Blog is a public legal resource and social commentary.

Let’s Get in Touch

Translate »