Team Approaches to Accessible and Inclusive Design: a Case study on the COVID Alert App and Healthca

[amazonv]

Team Approaches to Accessible and Inclusive Design: a Case study on the COVID Alert App and Healthcare Portal

Type: Breakout

Track: Design

Everybody has the potential to craft inclusive experiences when they talk to the people who operate & use services. This talk is a story of how a grassroots approach to accessibility and inclusion infiltrated the roles on Canadian Digital Service product teams allowed us to scale accessibility within rapid development of the Covid Alert exposure notification tracking app in just 45 days from start to launch.

 

We will talk about how organizations can prioritize accessibility and inclusive design and consider the points of intervention in the system to leverage better outcomes. We do this by exploring the space of possibilities and uncovering points of exclusion. We will explore the role and impact of service, interaction, and content design as well as their intersection with development, research, and product. Product is not linear. Design is not linear. Inclusion is a spiral of access points and off-ramps. Let’s go on a journey together and explore more inclusive service design and delivery.

 

[missed some of the start]

 

>> Thank you folks.  I would like to talk 

We are working in multi disciplinary team 

 

 

>> Screen froze.  

We had to start with the COVID alert app and trying to ensure we were 

making it accessible and inclusive from the start and had this launched 

in less than 45 days.  I am excited to dive in to behind the scenes and 

some of the key considerations to make it as accessible.  Government 

services should be simple for all.  That is what the mandate is, serving 

people better.  We do that through dedicated teams of people who are 

working towards the different levels of government to respond to needs.  

And so everybody has the potential to craft an inclusive 

experience.  When they talk to the people who operate and use the 

services, because at the end of the day we're all in some way, shape or 

form, an experienced designer whether the roles in which we're hiring 

for in the descriptions that come, how we staff and set up teams, the 

choices we make as an organization and structure all play key roles.  

So it is also important to know it is part of a system, so the 

main outcome was to help reduce the spread of COVID-19 by creating 

thoughtful privacy notification service that addressed as many barriers.  

As the pandemic started to roll out we got in to June and the government 

of Canada announced that it would create a nationwide exposure app.  So 

we got started with an open repository called COVID shield as well as 

apple Google frame work that was released on phones.  So I want to take 

a step back for one second and look at what is it that we're here to 

talk about.  Because there are a lot of considerations on what is 

exposure notifications, and it is not something around a year ago or 

five years ago.  Technology didn't exist being utilized in this way.  

There was a lot of concern about that, so I want to know what isn't 

COVID alert.  It is not contact tracing.  It is important to note that 

it values privacy and built with strong considerations and went through 

security iteration as part of its way of how we look at servicing 

people.  We needed to ensure that it was a reliable and consistent 

experience honest with people.  That is why it is also voluntary, so you 

can download, delete, disable Bluetooth which make it not work and gives 

people choices on what to do when they're using it.  So when we talk 

about moving in a quick manner to respond to public health crisis, 

launching something we haven't started before in less than 45 days is a 

big deal.  There are a lot of thing you may need to consider.  On the 

screen there are two pictures of the COVID alert app.  That is the first 

that allows you to select the language, where English and French are 

recognized.  And it is first screen once set up to let you know you're 

set.  

There are a lot of routes we can choose, different paths.  So we 

needed to pick some things that would guide how the service would work.  

It needed to work well, but also needed to be used by as many people as 

possible.  So that meant the service had to have some pillars that it 

would consider every step of the way and guide all the decisions.  

Essentially what happens here is that we needed a service that would 

detect and know if you had been around somebody with COVID-19 you would 

get a detection.  It needed to guide you what the next appropriate 

action was, what does the province ask you to do.  Because each province 

is governed differently.  And there was also availability, it must be 

available for as many as possible.  One of the early things that 

happened is that it only worked on cell phones less than 5 years old and 

only went to the I-Phone 6 S and that left 20 percent of the cell phone 

carrying population.  So there was a lot of advocacy and signals sent 

through policy and advisory councils that it didn't meet the needs.  

Within six weeks apple and Google released an update and now it is back 

to phones that are 7 years old, back to I-Phone fines which covers 97 

percent.  Then we had pillars that would guide everybody.  It needed to 

be used by as many people as possible, we needed awareness, and how it 

would help them in their communities and how it worked.  

There had to be trust, confidence the app would be private, and 

security.  There were a lot of considerations when you look at having 

more than 50 federal partners, 60 proVINGSal and more than 9 provinces.  

So we relied on having procedures documented, like how to improve a 

design change or hand off research insight to design iteration to 

implementation and verify that it met a need.  We needed to see where we 

are in the system, how would we design intentional inclusion and how 

many factors there are.  There is a diagram that represents the solar 

system, but has things like public health, Canada, health care workers, 

businesses, vaccinations, apple Google frame work, lock down, PPE, so 

COVID alert is just a small piece of that system and how it responds.  

But if there is a lack of PPE, there is a significant need because 

people need to be aware when to modify behavior or if an advisory 

council is pushing for advocacy to change a piece of technology then we 

can adapt.  We need to look at the connections in the system, what are 

the emergent needs, what are the support structures and what are the new 

rules.  So we went from having 0 users to 600-0000 within the first 

90 days of launching the app.  That meant there was a lot of diversity 

so we had to set up products and services to respond well to those 

needs.  When we started with design research, we grounded everything 

insight.  We did userability testing, user interviews, research that 

looked at whether or not somebody was going to address a need.  We 

designed inclusive designs where we served content, interaction, visual 

experience and service.  With development it centered around the 

security, reliability and accessibility on the app and policy to send 

signals around the enablers.  We also needed to make sure we had proper 

support and outreach so the awareness efforts, storytelling but also 

support activities because there is nothing worse than releasing 

something in to the wild expecting people to utilize it, it is 

responding to a health care need and then not having support.  It is a 

multi service product system, built in POT Canada's official languages 

and done that way every step of the way.  If it was written in English, 

it was written in French.  And there were five teams set up, working in 

tandem.  A mobile app team, health care portal team, server team, 

partnerships and support team.  That meant there was a lot of humans 

working on how this works and a lot of considerations to go back and 

forth to make sure we're dong things with intention that we want to make 

sure the partnerships are aligned, going through the same goals, looking 

to meet the same health care needs, looking to respond and use the 

service in a way that makes sense.  

There was also need for looking at an international collaboration.  

Because as this was happening, we were some of the first in the world.  

Right now I think there are more than one hundred COVID alert apps or 

notification apps but we needed to leverage the international community 

to work and learn from each other as we were in different phases.  So we 

worked with volunteers from the public health foundation.  We also 

started with Germany and Italy who were some of the first on market, 

policy needs, considerations they thought about.  We also did the same 

with Ireland.  We had a lot of collaboration and participation with the 

UK and the Netherlands and Canada presented together their COVID alert 

apps and how they were responding to needs and basically being able to 

leverage experiences.  The U.S. shared insights on how to share the API 

for using the one time keys provided by health care workers through 

somebody who has tested positive and how to set that up in different 

states.  In Australia and New Zealand we exchanged that information, and 

Mongolia started working on their own.  A lot comes from working 

research with real users.  On the screen I have an example of one of the 

phases of our research insights.  Our team took the time to look at the 

facets that makes somebody whole.  In this particular instance we had 

five participant, one was French, five was by link y'all and three were 

English speakers.  P what we want today know here was did the 

participant understand what inexposure notification was.  And then we 

had observation that enabled us to make decisions on the app.  We also 

created service blueprint.  The point for this was to be able to know 

what the awareness level was, actions, knowledge gaps, who were the 

owners, who was monitoring, did we test it to look at whether or not we 

were responding to the emotional and physical needs of people.  We asked 

what the health and well being and measurable outcomes of the work would 

look like in terms of impact.  This meant we were able to come over with 

some over arching insight that helped guide how we will move forward.  

We needed to be flexible for provinces because they were set up 

differently, minimize burden on workers who were already tasked, so it 

had to fit in their systems and consistent experiences for app users.  

That meant we would rely heavily on working well together.  We used some 

tools to help us.  One of those was get hub where we were able to work 

completely on the open, had people responding before the app was ever 

released trying to compile, build and respond to things that might be 

bugs or issues, but able to increase the visibility of accessibility 

issues.  Able to tag the priority level, able to say whether or not it 

came from somebody doing testing.  In order to make sure we knew all the 

time where we stood so that we can do this in a continuous way.  We also 

used the design software to help us work collaboratively.  This enabled 

our stakeholders, entire team, research developer, policy experts to all 

be in the same file and be making changes optimizing and suggesting new 

approaches.  And that allowed us to collaborate much more seamlessly 

given the amount of stakeholders.  We also used something liked 

T-R-E-L-O which was enabling us to keep track of the work we had in 

backlog, tag people, and detailing tasks and checklists that need to 

happen.  We had goals for sprint, tasks, what was being done right now, 

what was being reviewed and maybe why and what was done was able to be 

moved off to outside of the sprint to be validated.  

It also meant that we did a lot of meetings together, and they 

were imprompt TU but we also had guiding rituals.  We worked in two-week 

sprints and did retros.  It was important for us to discuss what E were 

doing and how to improve.  We were able to see blockers and respond to 

those needs so we were not stuck in a position where we had an unknown 

problem.  And that enabled a lot more clarity for people to move 

forward.  We needed that because we were navigating ambiguity.  For 

designer, I believe it means that we're able to hold opposing and 

different thoughts intention.  And that means we may not have the 

answers now and we will have to make decisions and have to root and 

ground them in something important that guides us.  We have to be 

prepared for the challenges that exist and constraints in knowing what 

we know right now.  So because there are many directions and 

possibilities that are open, we needed to find a pathway forward to the 

unknowns.  L that meant we were designing for resilience, we had to 

think about values, and ethos.  We had to identify how those assumptions 

were imbedded in practice and realign those assumptions.  Some of the 

ways were talk King about mind set and embracing curious mind set.  A 

random question may create an opportunity, so what if and what now?  

Consider multiple perspectives, people teams, coworkers, stakeholders 

and people using your service, because inclusive design and demands uses 

diversity to challenge what is possible, shifting perspective.  And we 

prompt an experiment because the road to success and change is paved 

with those small experiments and recognizing that struggle and embracing 

ambiguity allows us to move forward when we don't know where something 

will lead.  We're aim together break the cycle with intentional 

inclusion.  

On this screen I shared a graphic that is an adaptation from the 

book mismatched and how you recognize cycles of exclusion.  Why, when, 

who makes it, how it is made and what we make.  And who is going to use 

it?  But I have also added foresight here in terms of that whole aspect 

of so what?  How does this affect us, how does this change the future of 

what we're doing?  Have we learned to turn insight in to opportunity and 

how will we do differently what we know.  That was to be uncomfortable 

not knowing everything, identifying pain points and reduce the problem 

space by focusing and activating goals forward.  So we have a lot of 

opportunities -- as many as we allow ourselves to get better at 

intentional and be more intentional on accessibility.  We don't have do 

it all today or tomorrow.  What we do afterwards is critical to the 

culture change and outcomes for people.  So aim to ask powerful 

questions, provoke your thinking.  What is important about that for you, 

shift your perspective, how might this look from our user perspective, 

check assumptions, what assumptions are we making?  Challenge your 

beliefs, how else could this situation be interpreted?  And then take 

stock of what we know now and lead to curiosity to aim what is missing.  

There is great graphic on how to aim at ambiguity.  So learn from 

others, synthesize information, communicate deliberately to move between 

concrete and abstract.  And then when you can craft and build 

intentionally, you're designing the design work.  You need 

intentionality prompts, notice and reflection, able to identify and 

share and hold space STO do that work and process that so you're 

balancing quick action with thoughtful reflection.  

I have basically taken a design process here and a traditional 

prototype, but I have changed to emphasize and understand.  We can't 

always empathize with somebody's situation, we don't know what it is 

like to be them.  So hoping we will understand all the situations that 

somebody is in is not really true, but we can aim to understand.  And we 

can understand by knowing the power, the mental models, context, the 

partnerships, intention and identity.  We may look at people aging, 

mobile DWOISs, older technology, primary language may not be English.  

And then people that are maybe deaf or hard of hearing, low vision, 

blind, have episodic disability.  Maybe they have low bandwidth or 

mental health condition.  

So we need to make it a safe space for people to give feedback, 

and by being flexible on when how and whom, enables the trust and 

willingness of the engagement with the team.  In those retros I talked 

about that we did as retrospective, we would use things like slack and 

send direct messages and getting a pulse check if they felt they were in 

a safe space, if they thought they could share most thoughts.  And if 

they had a word or phrase that could share their experience on how the 

week was, the retro, if they were heard.  We give them multiple ways.  

They could do it aanonymous, slack, video call.  Or somebody else to 

give that feedback to.  That is the core of how we work together.  And 

then how do we facilitate the backlog.  What is it?  It is a list of new 

features, bug fixes, infrastructure changes, or activities the team may 

need to deliver for specific outcome.  They need to know they're 

understanding at every level.  If it is not in the backlog it is not 

going to be worked on.  

User rise findings and insight, and technical and design open 

source channels.  Then we have to look at how do we prioritize a backlog 

like that?  Is it public health priority?  Do we have proof the user 

needs this or are we solving the right problem?  How urgent is it?  And 

is this creating debt?  That means we have guiding principals, plus 

sources and prioritization equals a backlog.  How often do we do that?  

Every day.  

One of the reasons is to make it accessible for every one.  Some 

people ask why do you need to make it accessible for all?  In Canada it 

is pretty diverse.  22 percent of people identify disability, and more 

than two percent of the population don't speak English or French.  So 

service design becomes important and intersects with inclusion.  An 

example of after we released the app we had exposure windows because we 

had new information from epidemiologist and we could say you have been 

exposed on this date or this proximity and this is how long your 

exposure would be.  

This came out of some of the research insights and the fact that 

we were using a continued iterative approach to accessibility.  So at 

the onset we created an accessibility plan.  We started doing what we 

coined as an inclusive design review.  We did more than seven of them in 

the 45 days before launch, two after launch.  And because we realized 

the team was working so closely together, they decided to address 

service inclusion decisions, interaction decisions, without having to 

have it be separate.  More than 70 accessibility issues were answered 

before the launch and more after the lawn FRP.  There are less than five 

accessibility issues now, and we do manual and automated testing as we 

ship and deploy new things as well as when it is being built.  That 

continuous testing features and functionality help us recognize this is 

a team sport.  It is not all on developers, not all on designers or 

researchers, it is the whole collective of a team working towards that 

goal.  

And some of the things that enabled us to do that is to develop 

accessibility triage tactics.  Decisions that were being made, and the 

issues were tagged using labels to support that visibility and scope, 

but also empowered people to say I have a problem here, this needs to be 

fixed.  That enabled us to get prioritization and to not have that 

become something that fell off or got pushed to a parking lot and not 

achiefed.  

Some of those tactics were upstream, so we had worked and are 

working on a COVID shield.  And we were referencing upstream issues so 

they could pull the fixes back up.  We had low pry errorty issues, 

medium priority issues which do not impact functionality, and high 

issues which were critical.  This work flow was created to support 

people who were being on boarded and the accessibility of beta testers, 

new issues that needed to be addressed.  It kept access about visible 

every day.  It supported conscious decision making in terms of 

priorities for the team but also identified where we may need better 

resources and support structures.  

It also lead to some big wins for government.  We have a public 

accessibility statement and a full public accessibility report open.  

And a service design lens that was applied with accessibility touch 

points.  Our beta testing included more than 6 thousand users, some that 

self identified as having a disability.  We wanted to make sure that an 

accessible service would include all touch points.  When you call 

service Canada, there is a TTY number to call.  It comes over to the 

Canadian digital service if it is something we need to respond to.  

There is an e-mail, a get hub repo with open feedback channels, so 

different ways in which somebody can engage and it becomes multi modal.  

Because we were working on those multi display nare teams, we needed to 

make sure we were also looking at inclusive governance.  What is 

inclusive governance?  It is essential to long term sustainability 

because governance is only inclusive when the efficacy is measured on 

how well it engages all.  What is the service, the system, how does it 

affect uptake?  The design is a pillar of the accessible Canada act and 

there is a need for different approaches with different demographics and 

the apple and Google frame work means sometimes we're more reactive 

instead of proactive because frame work changes the DWIS.  So priority 

is determined and guided by health care needs and enables us to respond 

in an agile way to fix, build, implement, repeat and make things 

sustainable in terms of core functionality and sustainability.  

We are able to leverage that every day, and we advance responding 

to the diverse needs of the populations.  

Inclusive governance sends signals for availability, accessibility 

and adoption.  It allows us to better pivot to we can serve the 

populations better.  We need to recognize how the echo system performs 

and directly intersects.  L on this screen I have a map of the 

approaches we discussed today.  Design reviews, userability from diverse 

communities, open issue queues, testing and things like that.  It also 

highlights things we could have done differently or better.  One of 

those is earlier tech intervention around automated accessibility.  It 

would have been great to have that set up from the first time we set up 

the build and then having to work backwards.  

We needed to re-evaluate earlier with stakeholders to look at that 

from the day we started and how we would measure accessibility and 

inclusion.  What we ended up with is more than 11 thousand support teams 

in the last, 6 or 7 months.  Hundreds of articles about access and 

availability, because when it first launched it only went back to apple 

or I-Phone six S.  

An e-mail address has been KASHTHed more than 26 times, and 

highlighted a key need.  Over half were related to emergency response 

notifications that you get from cell phone providers about COVID in your 

area or amber alert.  We were able to reach out to the departments 

responsible for that to address other issues and it allowed us to 

leverage a more whole governance response.  Key areas we had to consider 

were the types of work that needs to be accomplished.  We looked at how 

we worked together, how we might have governance, but what is the actual 

app?  We started with content design and needed to know that trust 

depends on understanding.  People who downloaded the app needed to 

trust, and if not they weren't going to use it.  A lot revolved around 

the people and the people they were with and places they go.  Most 

people think about virus transmission and it is tied to people and 

places.  If the people trusted the app would not track them, we had to 

show that, that the app would record exposures without tracking 

movement.  In other words we had to change the way of thinking of how 

the app actually worked.  So there was some over arching principals.  

Inform just in time, give just enough detail, spread out the density of 

that meaning and write it like a real human.  That meant to be 

conversational, not to give walls but text as people needed it as they 

move through the different screens.  Then we also looked at 

illustrations.  So there was a lot of feedback that came in about how 

people perceived information and what to do when you're looking at 

something that might be a bit anxiety inducing.  Because we were 

striving to be approachable to as many people as possible, we needed to 

have the insights around this.  Those research insights allowed us to 

change the on boarding screens and add some illustrations.  Some 

examples would be what is on the screen now.  The ultimate goal for 

these designs was to show a range of diversity in order to allow every 

one to feel like they were included in part of this experience.  So we 

needed some principals, create a clear and mobile and friendly concept, 

friendly storytelling.  We needed to make sure whoever viewing the 

images didn't feel discomforted or distraction or burdened.  We wanted 

to make sure it was accessible.  And that it was more universal, simple 

graphics understood by as many as possible.  They may seem like large 

ideas, but they were guiding concepts on how we would explore and 

visually represent the information.  Showing diversity beyond hair, hair 

is one of the most wonderfully diverse thing about us but also one of 

the things that can have someone judge us really quickly.  So how do we 

show there's a lot of diversity in something without having it be 

represented just solely by hair.  We also thought of things like using 

minimum color pallet.  Part of that was the pigmentation of skin color 

and the way your eyes move to different levels of contrast.  So the lack 

of contrast leaves little detail for eye to see again, bouncing your eye 

back the people with different contrasting details, for example those 

with lighter skin, making them the focal point of illustration.  

We decided not to give lighter characters pigmentation.  The 

second is the screen in on boarding, about privacy being safe.  One of 

the things here was we wanted to make sure that people needed to 

understand the intention of the service, and if nay didn't it would lead 

to mistrust.  The consequences of not understanding or following the 

rules of the service transaction can lead to negative consequences and 

prevent them or us from meeting our goals.  How do people move through 

the flows of what we're representing, what is the experience when they 

go from one thing to another and how are they interacting with it?  And 

so when you open your phone the look at something like COVID alert, 

there is always a moment of anxiety or anticipation.  Receiving exposure 

notification will not be pleasant and there is no real way around that.  

Ultimately the right thing to do as designers is to be as gentle and 

supportive as possible and guide people through the experience they're 

going through in this pandemic.  This is why in addition to giving out 

110 keys, we created an alphabet that took out letters like W for 

whiskey and instead used WiFi.  We want today have a universal story.  

The main screen contains -- all the screens use a hand.  You have no 

exposure or all set is a thumbs up.  If you have the agency and autonomy 

to gather more comfort from what you're seeing versus something that 

might be a little more scary or alarming.  

When we use that, we wanted to make sure that the first motivation 

was to help the human rather than machine.  Instead of helping the 

machine communicate status to the human, we decided symbols would be 

used by the hand.  So as complex as it may be, what matters is how do we 

translate it to human language.  When there is nothing to report, it is 

a thumbs up says everything is well.  But if there is a potential 

exposure, there is a small stop sign with hand up to seek guidance.  The 

only deviation from that in the app is when we needed a role reversal 

and there was something wrong with the machine and the human needed to 

intervene.  If you had Bluetooth off, airplane setting on YUR phone, you 

will get a round circle with an explanation point that is red indicating 

that there is something that needs to happen in order for the 

application to work.  

And this particular screen goes through some of the flows of the various 

screens that were created.  The idea is that COVID alert is there to 

guide people as a collective but most importantly as human beings with 

feelings during a pandemic.  

Then we get to development, we have gone through what content, how 

illustrations may be ready, experience of interaction.  When we got to 

development we were late before we started but we wanted to make sure we 

had the space.  IOS 13.5 had the frame work, because Google and apple 

pushed that.  There was a lot of media coverage talking about delays, a 

significant amount of pressure that we have something and that we 

respond to needs.  And there was a lot of work to do.  The proof of 

concept on how we might do this worked so we needed to be able to move 

forward.  Unravelling somebody else's work is a big task.  We need today 

do that in a way that was meaningful.  So we coded it in the open.  We 

wanted to thank other countries that allowed us to stumble together and 

people can fact check our claims.  

When we tested the app, we built our app on the foundation of COVID 

shield.  That meant that there was already a series of volunteers 

supporting in the development.  We had beta testers who turned in to 

advocates and QA processing became second nature.  Manual testing still 

haunts us because it was one of those things where Android devices 

worked across a variety of different O-S installs and that meant the 

experiences was different.  

We wanted to think in out comes and impact and make sure everybody 

is aligned on goals and values.  Understand impact of actions and carry 

the weight together.  You have to let research and health care needs 

serve the product.  We were able to strive for that step every step of 

the way and bring other folks in that way.  Working in the open takes 

practice, you need strategy.  Getting your government to work in the 

open, you don't want the practice when it is large scale or public.  Be 

as timely as possible when responding to issues and set priorities, 

sharing code plus you shall shoes helps every one.  We have been able to 

look at others and discover common problem and bugs and helps us improve 

functionality and accessibility of the app.  What is next?  We're 

starting to look at how we do this in terms of product planning and 

accessibility and inclusive design work flow and sparked interest.  

Now we are looking at ways to create and grow.  What are your 

product accessibility goals, how will we measure success, who will 

participate on the team?  What are we missing?  What are the potential 

obstacles?  Do we know the costs?  And what do we have right now?  But 

it also lists tools and resources that teams can use, or look at testing 

that they may have had manual testing and they have done testing to 

affect people that had disabilities, cross browser, fire fox, Safari, 

did we check things for manual testing like keyboard or reading order or 

if the alt text fits the content?  

I will share this separately afterwards in two outputs.  One is 

mural and a Google doc that breaks it out.  

What we want to learn now is how to improve access to COVID alert.  

We are still doing ongoing research, so if anybody wants to reach out, 

they can.  We're trying to learn about experiences with bandwidth, 

phones and computers, how it intervenes with health care, concerns with 

the app and anything else.  We would love to hear from anybody about 

things like that.  And if you want to learn more about our process and 

work, you can check us out at digital dot Canada.  

 

>> What research did you need?  

 

>> Well, the first example only had participants and then an ongoing 

series.  Basically every week a dedicated researcher was doing new 

research.  What that enabled us to do was we decided before we would 

launch that we would do a small scale beta with friends and family to do 

snowball research to get a bigger ball and then that insight lead us to 

there is more to do, and we had a three week delay and a large scale bet 

that with 6 thousand people to get a much more impactful series of what 

are the things people are experiencing, what happens with different 

devices, are people understanding because that would be crucial to the 

efficacy.  

 

>> I design software, do you have any idea on how to contact medical 

professionals, those using electronic health systems to test products?  

Do you have any automated tools you may suggest?  

 

>> We have not used any automated tools to my knowledge.  We have tried 

to look at ways to use, especially being the government in our case.  

There are some organizations and companies out there that can help.  I 

believe fable and nobility do that research where they can find 

participants from a variety of different backgrounds.  There are ways to 

find that through, out sourcing.  

Often it is the ways in which you engage and how many places you 

are looking and where will you find those people?  Because if you're 

looking for them on Twitter and they're not there, you have too find 

them.  You have to have an alternate approach.  A lot of that helps if 

you can go through community support initiatives as well, but there is 

also the balance of burden on those in this type of timing.  

 

>> Makes sense.  Thank you.  When internationalizing the application, 

was sign language for deaf people a consideration?  

 

>> I would say yes.  But also how we would do that became an interesting 

conversation in how do we include everybody?  How do you include things 

like sign language in English and French in an application, what are the 

best ways to do that and I'm not sure we have come up with a good answer 

and something we would love to learn more about if somebody would like 

to reach out.  We have done some research with deaf participants, or 

deaf and hard of hearing, but right now the material that are online on 

the websites do have ASL in some of the video content, but not 

everything has been interpreted that way.  

It is definitely an area we can improvement 

 

>> Sure, if anybody has ideas.  

How did you reach out to your native populations?  Did you to work 

through cultural sit YAGs, native, trust?  

 

>> There is a lot of distrust and mistrust from indigenous experience 

especially with working with government and research.  We did look at 

the policy implications and the constraints of government and how 

government is supposed to do it, because there are ways we should engage 

those populations.  And I think for my knowledge right now the ways we 

have done that is through the community facilities located on reserves 

and things like that.  I am not sure of the numbers, but I know we did 

that capacity and it is an area that is difficult to ensure you get a 

good consensus of population.  

How inclusive you are in your approach, is it something you're 

able to allow them to lead more of the research on or are you in a 

position to do that.  And in this case, because we were responding 

quickly to critical needs, it was something that happened but didn't 

happen as frequently as maybe it should.  

 

>> Okay.  Can you explain how the federal approach was decided for this 

app?  In the U.S. it seems their only exposure notify KAGS applications 

if an individual state provides them.  

 

>> So that's one of the reasons in Canada that not every province and 

territory on boarded.  We have a similar situation where each province 

has their own health ministry, and because they have their own health 

ministry they have their own decision makers, people that have to be 

bought in to doing that.  The federal government decided they wanted to 

and we started with launching the just one province, Ontario July 31st, 

then Quebec, Newfoundland, Manitoba and then continued until we had nine 

provinces and one territory.  There are still two province that have not 

on boarded.  They those last two cannot use it when he will, but if 

somebody from Toronto flies to Vancouver and the person from Toronto was 

later diagnosed, they will get an exposure notification.  The one gap is 

that the one time key or thing that let's other people know they were 

exposed is given through the health ministry.  So it is a call, you have 

tested positive, here is your key to put in the app so there is layer of 

security and intentionality there which is why we have the health care 

portal.  Some provinces are doing that so they don't have an online 

system, and others are using API and grabbing the server.  So we had to 

use different divergent needs.  Because each state operates similar and 

some counties, it is difficult to mirror that approach.  

 

>> This is probably our last question, so thank you to everybody for all 

the questions and thank you for your time.  I see the application is 

built with reAK native.  Did the team face challenges with this like 

accessibility issues being present on one platform and not the other?  

 

>> Because we were taking a repo that already existed from somebody we 

had that dictated because some of the work was done.  On the development 

side we were already behind the curve, we had to go with that right 

away.  But it became really important and highlighted really quickly 

when when he started looking at adaptive technology that there were 

problems.  So we spent that six weeks, 45 days, with four testers from 

health Canada that were testing new iterations, new commits to the 

repos, new builds of the app that were being shared on Android and 

I-Phone.  We were doing it with simulated browser stacks at some points 

because there were problems with focus states and being able to read 

content that was underneath when a menu was expanded.  So there was 

definitely a balance to strike there and it became important for that 

accessibility.  If we had waited until the end, we wouldn't have be in 

the best spot.  

 

>> Thank you.  I will say thank you no everybody involved and enjoy the 

rest of axe con.  Thank you for spending this time with us.  

 

>> Thank you.