Persona – Amber, activist for inclusive education.

Amber

young female face with whispy hair eyes looking down

Age: 23 years

Course: Computer Science 2nd Year

Hobbies: Yoga, running, actively taking part in lobbying for inclusion

Background:

Amber loves her yoga and running, excelling at school where she enjoyed an inclusive educational setting despite her severe bilateral hearing loss.  She is a hearing aid user and depends on lip reading (speech reading) and Sign Language, but speaks relatively clearly.  In fact, her communication skills are remarkable, thanks to the early and continuous support she received from her parents, both mathematics professors.  Nevertheless, being at university was quite a challenge in the early months.  She admitted to feeling isolated, but used her yoga and cross-country running activities to get her through.  She explained that the psychological and emotional feelings were almost more challenging than dealing with lectures, tutorials or trying to access support, plus she missed her family and friends in the Deaf community.  It was the constant need to explain to people why she had not heard them or put up with their replies “Oh it doesn’t matter, it’s not important” if she queried what had been said – her disability was non-visible The process of disambiguation can be very tiring and as Ian Noon said in his blog

“Processing and constructing meaning out of half-heard words and sentences. Making guesses and figuring out context. And then thinking of something intelligent to say in response to an invariably random question. It’s like doing jigsaws, Suduku and Scrabble all at the same time.”

Amber joined the Student Union and found fellow activists who became friends as well as lobbyists for disability awareness and more inclusive teaching practices.

Those who could use sign language were rare, although help was available from interpreters during lectures and at other times, from note takers.  Sometimes, Amber depended on fellow students’ notes, because it was hard to write, as well as concentrate on the lecturer or interpreter.  Giving a lecturer her FM transmitter and microphone helped, especially when they turned away to write on a board or there were discussions and noise levels were raised.  When lecture capture was in use, Amber always hoped the Automatic Speech Recognition (ASR) would provide good quality closed captions and subject specific words would be transcribed correctly.   During the COVID-19 pandemic, Amber used text-based Q&As and comments when events happened in real-time with captions or subtitles and transcripts if available.  Downloadable summaries and slide notes provided details missed on the video recording. Help and documentation about caption position adjustments, the text size and colour plus other settings for the conferencing system and access to a glossary of important subject related vocabulary has also been invaluable to aid comprehension.

Main Strategies to overcome Barriers to Access

Structured content 
Signing on videos – Personalising alternative format content
Multifactor authentication
Document Accessibility
Audible Error or Alert Messages must be inclusive with alternatives
Multimedia content needs captions that are accurate and synchronised with the audio.

Multifactor authentication Amber has found that the best way for her to overcome the multifactor authentication issues is to use text messages (SMS) with a onetime password (OTP) plus a smart phone app.  She has also found that finger biometrics on her phone helps with authentication or using an email system.  She has a card reader for her bank but does not use speech recognition or voice calls for any verification, as she is never sure if she is going to get an accurate result or hear everything. (Web Content Accessibility Guidelines (WCAG) 2.2 Success Criterion 3.3.7 Accessible Authentication)

Multimedia content needs captions that are accurate and synchronised with the audio. Video and audio output complement so much of web content and Amber depends on good quality alternatives to make services both usable and accessible. It is not just the content, but also the way players work and the fact that any audio needs to be controlled and not start automatically.  WCAG Guideline 1.2 – Time-based Media discusses content challenges and User Agent Accessibility Guidelines (UAAG) 2.0 provides technical details for developers such as Guideline 1.1 – Alternative content. The Authoring Tool Accessibility Guidelines (ATAG) also for developers and designers include the following guides related to those who have hearing impairments:

Signing on videos – Personalising alternative format content.  Being able to adjust the position of the window with interpreter signing and/or captions may be important when content is filling the screen in different places as seen in the YouTube lecture on Beginning Robotics provided by University of Reading.  Further guidance is available in their blog on Personalisation of viewing

Structured content is important for everyone and helps Amber when she needs to pick out key topics.  So clear headings and sub headings, use of white space and links to summaries, especially if they are related to video and transcript content, so she can see what is relevant to her needs.   So regions, headings and lists all help to make a content more understandable.

Audible Error or Alert Messages must be inclusive. Notifications need to be available as very visible alerts as well as being audible and, on a phone or electronic watch, this can also be via vibration.  

Document accessibility is important whether it is related to providing summaries and clear language or heading structure and bullet points, the aim is to make documents usable as well as accessible.   WebAim have an easy to follow set of instructions for making ‘Accessible Documents: Word, PowerPoint, & Acrobat’.

Key points from Amber

“I am a very visual person and I like to see a detailed process as a diagram or image – step by step.  I also really like having PowerPoint slides with all the main points…”

Interviews with Three Deaf Software Engineers in Bay Area (USA) Facebook video

TPGi provided a very helpful set of articles around an interview with Ruth MacMullen, who is an academic librarian and copyright specialist from York in the UK, called  “Sounding out the web: accessibility for deaf and hard of hearing people”  Part 1 and Part 2

The latest version of the iPhone operating system iOS 14 offers more accessibility options and personalisation with increased use of visual and vibrating alerts and further support for hearing aids

The “PacerSpacer” – Simplicity in Controlling the Pace of Presentation Software (like PowerPoint) “One of the most important things you can do for Deaf/HH audience members when using presentation software such as PowerPoint is to allow sufficient time for them to read the slides before you begin talking. For Deaf/HH individuals, this means allowing time for them to stop watching the interpreter or you, switch their attention to a slide, and then return their attention to either the interpreter or you. With an interpreter, even more time is required since there is a lag time between what you say and the signing of that message.” DeafTec, Rochester Institute of Technology (USA)

Persona – Eum, a would be computer scientist who needs magnification

Eum

Full face view of Eum of Asian origin with dark heair and eyes

Age: 21 years

Course: Computer Science 1st Year

Hobbies: coding, cooking and travelling

Background:

Eum developed glaucoma meaning that as his peripheral vision has deteriorated and his central vision has become foggy, he spends more time zooming in to read text rather than zooming out. He has had to depend more and more on assistive technologies to help with his computer science studies, having changed his plans that were to become a chef.  Eum loves cooking, but the stress of a restaurant kitchen coping with lots of unexpected obstacles became too challenging.   As time has passed he also found he does not always recognise faces or signs in the distance.  He tends to trip up on curbs or steps, especially when it gets dark or there is low lighting, however, this has not stopped his love of exploring new places when on holiday with family. 

Reading can be difficult without careful targeted magnification and he tends to set up his two computer screens with good contrast levels for black on white text or other high colour contrast options, whilst trying to reduce reflections from windows and lights. At times he finds it easier to move the text in front of his eyes rather than scanning across text because of his reduced field of vision and the depth of focus on individual sections of text needed.   Time is often an issue as assignments tend to take longer to complete when reading only a few letters in one glance.  Eum uses a magnification cum screen reading program on a Windows machine, so that he can access various arrow heads, crosshair and line cursor sizes with focus tracking.  The latter means that items are magnified as he moves across and down the screen.  A high visibility keyboard helps, as does his portable magnification stand for reading paper-based materials. 

Although Eum uses audio books on his tablet for listening to novels, he prefers to read academic papers and notes on his computer, where he can add annotations.   Messaging and emails using his smart phone are quicker with speech recognition and text to speech output, although Eum does not depend totally on the screen reader technology for navigation.   He has learnt over time where items are to be found and is very disciplined about how he personalises his desktop and filing system on all his devices.  As a computer scientist in the making, Eum has become adept at changing browser settings with ad blockers and other extensions, but this does not compensate for the clutter he finds on many web sites.  He becomes frustrated when developers fail to realise the importance of avoiding overlaps and disappearing content when sites are magnified.  He feels scrolling horizontally should not be necessary on websites especially when form filling, as this is a particular challenge with text fields or modal windows that go missing or where there is no logical order to the layout.

Main Strategies to overcome Barriers to Access

Controls are labelled
Keyboard Access for navigation
Multifactor authentication
Accessible Error or Status Messages
Image alt text descriptions
Consistent Heading order
Audio Descriptions
Document accessibility

Multifactor authentication for password verification.  Eum usually manages the initial password and use of mobile authenticator apps, SMS or knowledge-based challenge questions better than the grid style image or Captcha options. Time can be Eum’s real problem if he has to cope with different types of authentication where items disappear before he has had a chance to memorise or note down the code.  He cannot use retina-scan technology, but finger and speech recognition work well.  (Web Content Accessibility Guidelines (WCAG) 2.2 Success Criterion 3.3.7 Accessible Authentication)

Keyboard Access for navigation helps Eum as he has memorised many short cuts when surfing the web and finds it easier than mouse use when he has to track the arrow head or cursor.  He depends on web pages appearing and operating in predictable ways (W3C WCAG Predictable: Understanding Guideline 3.2).

Allowing content to reflow on Zoom.   Eum reads magnified content using large fonts that need mean a page has to reflow vertically (as a column) and remain fully legible and logical without the need for horizontal scrolling (W3C WCAG 1.4.4 Resize text (Level AA)). User agents that satisfy UAAG 1.0 Checkpoint 4.1 allow users to configure text scale.

Maintain logical order of Form ControlsEum benefits from the logical order of form controls with labels close to the fields to which they relate so that the flow is vertical on magnification rather than horizontal.  Dividing forms into small sections when they are long or complex with clear headings really helps.  However, careful checks need to be made with borders of buttons etc so that they do not overlap other controls or disappear from the screen with high levels of magnification.  

Controls need to have good contrast, shape and colour. Text needs to be distinct. The importance of contrast levels for all aspects of a service cannot be stressed too highly and yet it is often one of the items that fails in accessibility checks. Webaim provide clear guidance Understanding WCAG 2 Contrast and Color Requirements.  It is also vital that text and images of text are distinct from their surroundings – WCAG SC 1.4.3 (Contrast – Minimum)

Elements should have more than one visual cue. Eum finds it really helps when he is selecting features on a web page to have icons, colour as well as text such as when there is an alert.  Underlined text should just be used for hyperlinks.  Links and other actionable elements must be clearly distinguishable.

Changing Content without warning.  Because Eum uses assistive technology, whether it is his magnification software or speech recognition, he finds that any visual changes that occur to the interface or actions that happen without the page refreshing can cause problems.  He needs to be notified about the changes W3C WCAG 2.0 Name, Role, Value: Understanding SC 4.1.2 and there is also the need to Understand Success Criterion 4.1.3: Status Messages.

Responsive Design for tablet use.  Eum likes to use his tablet, but has to depend on its built-in access technologies and the responsive design for accessibility provided by the developers of web services as well as their customisable options.  He relies on a consistent layout throughout a web service with sufficient space between interactive items such as buttons.   A11y project mobile and touch checklist for accessibility.

Captions, Audio Descriptions and use of transcripts.  Videos can be very tiring to watch and so Eum uses captions with high contrast colours to make them stand out more against the background of the video images, using highlighter systems to mark key points in the transcripts and depends on audio descriptions if scenes are not well described in the commentary on the video.

Document accessibility whether online or as a PDF download or other format, Eum needs to make changes to suit his narrow field of vision.   WCAG Understanding Success Criterion 1.4.8: Visual Presentation has some useful guidance.  WebAim also have an easy to follow set of instructions for making ‘Accessible Documents: Word, PowerPoint, & Acrobat’.

Key points from Eum

“Allow for magnification with zooming in to read text in small chunks, taking care to make visual presentation as logical as possible for vertical scrolling preferably in one column layout”

Designing and Coding for Low Vision” featuring Mallory van Achterberg is a really helpful YouTube video where he says ‘If I cannot see it, don’t let me tab to it” plus lots of other helpful comments, as a coder who uses magnification.

There is a useful document called  “Accessibility Requirements for People with Low Vision” created as a W3C Editor’s Draft (04 November 2021)

Multifactor Authentication – Cognitive overload?

smart phone face recognition, code, finger print and numeric password screens.

Recenty the FBI warned that cyber thieves are finding ways around multifactor authentication protections. We may not own crypto accounts, but universities across the UK have implemented forms of two or multi-factor authentication to protect our accounts and we are grappling with strategies to ensure accessible forms for verification are available. The W3C WCAG Cognitive and Learning Disabilities Task Force has published an Accessible Authentication (User Story) to illustrate the need for the cognitive function test mentioned in the previous blog. It also highlights how memory impairments and difficulties with executve function can make MFA a challenge as well as time constraints.

There was a Twitter stream in February 2021, that highlighted more reasons why some of the MFA choices offered may not be helpful to individuals with Autism and Attention Deficit Hyperacive Disorder (ADHD). Devon Price’s insight into the frustrating nature of these extra levels of security illustrates the cognitive overload that can occur when several tasks have to be completed with multiple devices or systems.

key fob

If you do not have an external code device, the choices tend to centre around the use of a personal mobile phone. Having checked the websites of 50 universities it appeared that 42 (84%) advised students to use an authenticator app – usually Microsoft, that also offers a Self Service Password Reset. Other options were the Authy, Google, Sophos or Duo apps. All the university systems still require the user to remember passwords with the extra verification and then encourage a back up option via SMS, a phone call and some mentioned a landline or email. Only 4 universities offered a choice of two authentication apps and just 7 mentioned the use of a fob or external device, although one said this would not to work with Virtual Personal Networks. Preference for Authy as an alternative was mentioned in a question about the Microsoft Authenticator, as this can be used on a desktop computer. At the time of writing Microsoft Authenticator instructions do not mention a desktop verification option to their MFA.

Some universities in the small study used the Microsoft instructions, but when searching for support it took well over 3 clicks, to find out about the authentication options offered by 14 out of the 50 universities (28%), 9 of these websites either had no information or required a log in. This meant that a new student may have no way of preparing for this aspect of registration, although all of the websites had good connections to their student support or IT services.

Only one university appeared to depend on a memorable word for verification and the use of authenticator apps usually meant that the code could be used without an internet connection or a connection to a mobile network provider. This does not mean that copying a code results in successful verification on all occasions.

Where concentration or attention is an issue, as may happen with ADHD, the problem of copying codes from one device to another can become worse as more attempts are made. Too many tries leads to lockout, desperation and yet another feeling of failure, let alone time wasting and other more severe consequences, if the actions are related to banking. Actions that involve recognition and copying qualify under the W3C WCAG cognitive function test as requiring an alternative method.

Using biometrics is often considered a good alternative, but it is not always easy to get facial recognition to work on phones if you are blind or have dexterity difficulties and some individuals really do not like sharing their facial image. The UK National Cyber Security Centre admits that fingers prints are not always recognised if people have been working in some industries or you are elderly and even a skin condition can cause problems. Also not all devices offer the chance of using finger recogntion.

In summary Making Content Usable for People with Cognitive and Learning Disabilities suggests four options:

  1. Web Authentication: An API for accessing Public Key Credentials [webauthn-2].
  2. Single Sign-on (SSO) that allow users to access many sites with a single login (federated login).
  3. Two step authentication with Bluetooth links (no copying).
  4. Quick Response Codes (QR Code).

The use of authentication apps, with a set up of 5 additional choices was offered by 6 of the 50 universities reviewed, so they provided more than the above options, plus the ability to use a helpline. When checking the helplines it became clear that there were often rather a lot of questions surrounding the Microsoft Authenticator, as evidenced by the University of Hertfordshire’s comprehensive set of answers. So It appears there is still much to do to make the process more inclusive.

Multifactor Authentication update WCAG 2.2

laptop with login and password

On January 13th, 2022, a W3C Editor’s draft of the Web Content Accessibility Guidelines (WCAG) 2.2 was published on GitHub. Among several updates and new items, it includes processes for making multifactor Authentication more accessible and easier to use. Systems for auto-filling are allowed, as well as copy and paste, so that one does not have always depend on remembering passwords. Email links and text messages are included for those happy with using other applications and devices.

This is welcome help for aspects of multifactor authentication that were described in a previous blog, even though the requirement is not at the hoped for top level. However, it has been set at Level AA, so hopefully this new Success Critera will still be offered by web services later this year. As was mentioned in August 2021, passing the check is based on overcoming what is called the cognitive function test

“A task that requires the user to remember, manipulate, or transcribe information. Examples include, but are not limited to:

  • memorization, such as remembering a username, password, set of characters, images, or patterns. The common identifiers name, e-mail, and phone number are not considered cognitive function tests as they are personal to the user and consistent across websites;
  • transcription, such as typing in characters;
  • use of correct spelling;
  • performance of calculations;
  • solving of puzzles. ” (WCAG 2.2)

It should be pointed out that this draft has yet to be approved, but WCAG have set June 2022 as the date for publication.

As an aside, there is no mention regarding the impact of biometrics (such as facial or finger print recognition) in the WCAG document, which can also be used to support access to web services, but are not available on all devices. These systems do not suit all users, and if passwords are not used as part of a login process these could present another type of barrier.

Time-based one-time passwords (TOTPs) can also cause problems when they have a very short period of use (30 seconds) and a person may fail to complete the action several times and then has to take a break. A January 2022 review by PC Mag UK highlighted the fact that Authenticator apps can offer better security, when compared to text messages (SMS). Some have desktop options that may also be more accessible.

Moving on withTranscripts

Laptop and notepad on the laps of students in a lecture

Over the years researchers have shown how it is possible to have live interactive highlighted transcripts without character or line restrictions, such as is needed with captions. This is only possible when using technology, but with many more students using tablets, mobiles and phones during lectures it is surprising to find how few lecture capture systems offer this option.

It has been shown that physically writing notes by hand can aid retention and using laptops etc in lectures means there is access to all the other distractions such as social media and emails! However, having the availability of a transcript that provides interaction allows for key points to be selected and annotation improves retention for those students who find it hard to take notes whether by hand or using technology (Wald, 2018).

Systems that also offer transcript annotation linked to the presentation slides, intergrated with the ability to make personal notes alongside the synchronised text, are hard to find. Ways to correct words as you hear or see them, where there are subject complexities can also be difficult.

As was described in our last blog it is clear that all the corrections needed, tend to be measured by different forms of accuracy levels, whether it is the number of incorrect words, ommissions and substitutions. Further work on the NLive transcript has also shown that where English is not a first language those manually making corrections may falter when contractions and conditional tense are used and if the speaker is not a fluent English speaker, corrections can take up to five times longer (according to a recent discussion held by the Disabled Students’ Commission on 6th December).

Difficulties with subject related words have been addressed by CaptionEd with related glossaries, which is the case with many specialist course captioning offerings where companies have been employed to provide accurate outputs. Other companies, such as Otter.ai and Microsoft Teams automatically offer named speaker options which is also helpful.

Professor Mike Wald has produced a series of interesting figures as a possible sample of what can happen when students just see an uncorrected transcript, rather than actually listen to the lecture. This is important as not all students can hear the lecture or even attend in person or virtually. It is also often the case that the transcript of a lecture is used long after the event. The group of students he was working with six years ago found that:

  • Word Error Rate counts all errors (Deletions, substitutions and insertions in the classical scientific way used by speech scientists): WER was 22% for a 2715 word transcript.
  • Concept Error Rate counts errors of meaning: This was 15% assuming previous knowledge of content (i.e. ignoring errors that would be obvious if student knew topic content) but 30% assuming no previous knowledge of content.
  • Guessed error rate counts errors AFTER student has tried to correct transcript by them ‘guessing’ if words have errors or not: there was little change in Word Error Rate as words guessed correctly were balanced by words guessed incorrectly (i.e. correct words that student thought were incorrect and changed).
  • Perceived error rate asks student to estimate % errors: Student readers’ perception of Word Error Rate varied from 30% – 50% overall and 11% – 70% for important/key words: readers thought there were more errors than there really were and so found it difficult and frustrating.
  • Key Errors (i.e. errors that change meaning/understanding) were 16% of the total errors and therefore would only require 5 corrections per minute to improve Concept Error Rate from 15% to 0% (speaking rate was 142 wpm and there were approx 31 errors per minute) but it is important to note that this only improves the scientifically calculated Word Error Rate from 22% to 18%.

This is such an important challenge for many universities and colleges at the moment, so to follow on from this blog you may be interested to catch up with the transcript provided from the Disabled Students’ Commission Roundtable debate held on 6th December. One of the summary comments highlighted the importance of getting the technology right as well as manual support, but overriding this all was the importance of listening to the student voice.

Finally, if you ever wonder why speech recognition for automated captioning and transcription still fails to work for us all, have a look at a presentation by Speechmatics about AI bias, inclusion and diversity in speech recognition . An interesting talk about using word error rates, AI and building models using many hours of audio with different phonetic structures to develop language models that are more representative of the voices heard across society.

Guidance for captioning rich media from Advance HE (26/02/2021)

Transcripts from Captions?

young person looking at computer for online learning

The subject of automatic captioning continues to be debated but Gerald Ford Williams has produced a really helpful “guide to the visual language of closed captions and subtitles” on UX Collective as a “user-centric guide to the editorial conventions of an accessible caption or subtitle experience.” It has a series of tips with examples and several very useful links at the bottom of the page for those adding captions to videos. There is also a standard for the presentation of different types of captions across multimedia ISO/IEC 20071-23:2018(en).

However, in this article transcripts are something that also need further discussion, as they are often used as notes gathered from a presentation, as a result of lecture capture or an online conference with automatic captioning. They may be copied from the side of the presentation, downloaded after the event or presented to the user as a file in PDF/HTML or text format depending on the system used. Some automated outputs provide notification of speaker changes and timings, but there are no hints as to content accuracy prior to download.

The problem is that there also seem to be many different ways to measure the accuracy of automated captioning processes which in many cases become transcriptions. 3PlayMedia suggest that there is a standard, saying “The industry standard for closed caption accuracy is 99% accuracy rate. Accuracy measures punctuation, spelling, and grammar. A 99% accuracy rate means that there is a 1% chance of error or a leniency of 15 errors total per 1,500 words” when discussing caption quality.

The author of the 3PlayMedia article goes on to illustrate many other aspects of ‘quality’ that need to be addressed, but the lack of detailed standards for the range of quality checks means that comparisons between the various offerings are hard to achieve. Users are often left with several other types of errors besides punctuation, spelling and grammar. The Nlive project team have been looking into these challenges when considering transcriptions rather than captions and have begun to collect a set of additional issues likely to affect understanding. So far, the list includes:

  • Number of extra words added that were not spoken
  • Number of words changed affecting meaning – more than just grammar.
  • Number of words omitted
  • Contractions … e.g. he is – he’s, do not … don’t and I’d could have three different meanings I had, I would, or I should!

The question is whether these checks could be included automatically to support collaborative manual checks when correcting transcriptions?

Below is a sample of the text we are working on as a result of an interview to demonstrate the differences between three commonly used automatically generated captioning systems for videos.

Sample 1

Sample 2

Sample 3

So stuck. In my own research, and my own teaching. I’ve been looking at how we can do the poetry’s more effectively is one of the things so that’s more for structuring the trees, not so much technology, although technology is possibleso starting after my own research uh my own teaching i’ve been looking at how we can do laboratories more effectively is one of the things so that’s more for structuring laboratories not so much technology although technology is part of the laboratoryso stop. In my own research on my own teaching, I’ve been looking at how we can do the ball trees more effectively. Is one thing, so that’s more for structuring the voluntary is not so much technology, although technology is part little bar tree

Having looked at the sentences presented in transcript form, Professor Mike Wald pointed out that Rev.com (who provide automated and human transcription services) state that we should not “try to make captions verbatim, word-for-word versions of the video audio. Video transcriptions should be exact replications, but not captions.” The author of the article “YouTube Automatic Captions vs. Video Captioning Services” highlights several issues with automatic closed captioning and reasons humans offer better outcomes. Just in case you want to learn more about the difference between a transcript and closed cations 3PlayMedia wrote about the topic in August 2021 “Transcription vs. Captioning – What’s the Difference?”.