Exploring the embedding of accessible image descriptions into image metadata

Time to move into images as part of the accessible package we can offer students when working online! If you are a Graphic Designer or Photographer using tools to embed accessibility tags please check what I am saying makes sense!

Many in the world of digital accessibility know the work of W3C WCAG and image accessibility, and are used to adding alternative text and long descriptions for informing users about the contents of an image, diagram, photograph etc, in particular for screen reader users. These tags are added by those who upload the image to a web page, document and other publications. But what about enabling the designer or photographer to add the ‘alt text’ and ‘long desc’ to their image as they save it? This may not suit all situations, but it has the potential to ensure accessibiity ‘metadata’ (data about the image in this case) is always in place when sharing takes place. The data can be adapted later if necessary and those uploading images can still add tags, if the original metadata cannot be read by certain screen readers or applications.

My journey into improving image accessibility all started when i wanted to add some metadata to pictographic symbols and I was exploring how to work with accessible Scalable Vector Graphics (.svgs) in CorelDRAW, because they are much more responsive to size changes and Deque had highlighted their accessibility advantages and most browsers now support the .svg file format

graphics designer working at a desk on drawings - image metadata wth URL, name size etc but no alt tag or description.
https://jimpl.com/ offering online image metadata information – No list item or property telling us that there was no alt text or description, just no location!

I needed to learn why I was failing to find a way of achieving the embedding of additional accessibility metadata and Chris OShea from PPA Training introduced me to a set of editable properties that are available and then shared the link to Adobe Bridge to enable their applications to carry the accessibility tags. The secret is to find a format that is part of a recognised standard for example the Extensible Metadata Platform (XMP) standard so this can also be machine read by various digitial asset management tools.

In 2021 The “International Press Telecommunications Council IPTC’s Photo Metadata Standard included the two essential properties: Alt Text (Accessibility) and Extended Description (Accessibility).” The IPTC blog announcing this news said that

“IPTC’s new accessibility properties will make it easier for platforms and software to comply with WCAG requirements and deliver images that are inclusive for everyone. Embedding accessible image descriptions into the photo metadata will make it possible for alt text and extended descriptions to travel wherever the image goes on the web or in books or other documents provided as EPUBs.”

IPTC October 27th 2021

However this is not going to happen overnight because Chris and I discovered when testing the procedures, that not all software companies allow the accessibility metadata to be added to their graphics packages in a way that can be read by a screen reader. Richard Orme, CEO Daisy Consortium, kindly got in touch about his paper on “Making use of IPTC alt text accessibility metadata” where I learnt that at the moment the use of the ExifTool by Phil Harvey is the stepping stone that we need!

AVPreserve Exiftool Tutorial Part 1 on YouTube (22 Nov 2013, 4.45 mins) https://youtu.be/CWcMrAfhlKI

Exiftool is not a new tool and has been used by those setting up photographic repositories for many years and neither is the discussion about using it for adding accessibility metadata as the NCAM Potential Use of Image Description Metadata for Accessibility paper (2011) illustrates. A 2021 Exiftool set of instructions by Chris Blackden describes how metadata can be seen by everyone and removed or added. There is a very helpful video, but as yet he does not describe the addition of alt text and long descriptions.

So for those wishing to try Exiftool with a set of command lines, Phil Henry has examples on his Exfitool pages and Richard Orme has offered examples for adding the accessibility metadata

ExifTool command line utility

Rename the executable to exiftool for command line use

To set metadata use:

exiftool filename -AltTextAccessibility=”Your alt text here.”

exiftool filename -extDescrAccessibility=”Your extended description text here.”

To read the metadata use:

exiftool filename -AltTextAccessibility

exiftool filename -extDescrAccessibility

Hopefully, soon all graphic design software packages will include the additional properties for accessibility metadata and digital asset management tools will support the IPTC standard, so that users of assistive technology such as screen readers and text to speech apps will be able to find the accessibility tags when available!

It all seems much more complicated than I first thought whilst Artificial Intelligence and machine learning have moved the goal posts into new realms of digital image recognition. However, just allowing an image to be saved with embedded accessibility information did not seem such a knotty problem when I started on the journey!

A tangle of wires on a telegraph pole

Multifactor Authentication types across 50 universities

When considering the different types of Multifactor Authentication (MFA) it is clear that many could be a challenge for students with a wide range of disabilities. However, when you add the use of assistive technologies and customisation or potential personalisation the barriers begin to come down. That is as long as the actual website or app hosting the required verification of a sign up or log in is accessible.

With these caveats in place it seemed that as long as students were provided with at least three or more choices it would be possible to navigate MFA. That thought led to a mini survey of around a third of the universities in UK to see what was on offer.

graph of MFA choices in 50 universities
Vertical axis has the MFA options and the horizontal axis is the number of universities offering that type of option

Several universities offer a password as their main login method and then additional security for certain more sensitive areas. 42 out of 50 universities offer apps, but only two apppear to provide 2 options for the type of app, such as Microsoft and Authy on a desktop, which can be very helpful for assistive technology users who do not have smart phones or find their desktop AT easier to use.  8 universities offer hardware tokens and 6 offer at least 5 options but 9 had no alternatives that could be easily found and 14 universities made searching for support difficult by not having easy to reach information pages.

Microsoft authentication app, a text message to a mobile phone or a call to either a landline or mobile, were the most common verification methods after a login email and password had been generated. 

So in summary…

  • many students have limited options if they do not want to or could not use the Microsoft Authentication app or do not have a smart phone.  
  • there are rarely more than two options if using an app is not possible and one includes the use of a landline, which may not always be possible in a college or university setting
  • it often took more than ‘three clicks’ or selection choices to reach any supporting materials and these rarely mentioned the use of assistive technologies.  However, there was usually a contact form or email address available.

Personas

Featured

We have created 6 computer science student personas who need to use different ways of accessing online materials and elearning systems to highlight some of the issues encountered. In some ways these personas aim to fit into the NNgroup definition in their article on Personas vs. Jobs-to-Be-Done. Page Laubheimer in 2017 said “Jobs-to-be-done focus on user problems and needs, while well-executed personas include the same information and also add behavioral and attitudinal details.”

multiple flavours in spoons

We hope our personas are ‘well executed’ and the features mentioned relating to accessibility and usability will help support individual preferences when using the web. However, there may be overlaps and there will be techniques and criteria that you may feel are missing. So it is important to think of the content as just being a flavour of what can be offered when creating inclusive content, design and development. Follow the links for more information about individual criteria. Content will also go out of date and need checking, so do contact us if you spot any issues.

Each Persona has a video relevant to particular issues that might be encountered when web services fail to be accessible and cause barriers. Matt Deeprose and the team from the University of Southampton have provided a wonderful explanation about several aspects of web development in a video ‘The impact that high quality mark-up can have on accessibility, performance, and discoverability‘. Matt’s Github example web page of how he has used the code he discusses, also has the video transcript and links to many other resources. A good way to start on the accessibility, usability and inclusion journey.

Persona – Umar enjoys video games and using his Xbox

Umar

face of a man with dark hair, dark eyes and stubble.

Age: 31 years

Course: Computer Science 2nd Year

Hobbies: video games, travel, family and children

Background:

Umar used to spend all his time on is computer which has been adapted to suit his needs as he really enjoys interacting with friends online playing video games.  Now, his family takes up much his time outside university and they often travel between his home country and the UK.  

Umar has always worked in universities and was a lecturer before deciding to take up computer science as a second degree.  Formally linguistics was his main interest and he taught students English in his home town and often helped out with translations from Arabic.  He grew up in a household that encouraged him to compete at school in all activities despite his cerebral palsy that made him appear very uncoordinated, affected his mobility and dexterity. 

Umar has always spoken rather hesitantly, but relatively clearly, so lecturing was not a problem, although speech recognition never really worked for him.  He also tried eye gaze in order to access his computer, but found it very tiring because keeping his eyes fixed and still on an object was difficult.  So Umar largely depends on keyboard short cuts, predictive software, with built in macros using autohotkey, filter keys and a programmable position joystick.  Recently he was also able to afford an Xbox Adaptive Controller that made all the difference to the speed at which he could play games once it had been set up to suit his preferences with his mixed access technologies depending on the time of day and feelings of tiredness.  

Keyboard access still takes time, but as long as Umar has his specialist expanded keyboard and uses his own computer he can work successfully along with his fellow students.  However, where barriers do occur such as being locked out due to timeouts, with for instance online banking or secure applications, his patience can be sorely tested. 

Umar needs to take breaks but finds that additional ergonomic aids such as a good chair, two raised monitors linked to his laptop and the spring back keys on his keyboard, help to prevent some of the effects of fatigue.   In the past he used his phone with a Bluetooth connection to his keyboard, so that he could type messages, but he has found several companies now offer better links to apps, his joystick and other assistive technology devices such as Tecla.  He also finds adapting the phone notifications, so they appear automatically stops him worrying about missing events and larger fonts and app icons have also helped.  Bigger buttons mean there is more likelihood the touch screen can be used. Umar has also tried other accessibility options that are available on both the Android and iOS systems.  

Strategies to overcome Barriers to Access

Make dynamic content accessible 
Keyboard Access for navigation -  heading, links and buttons etc
Multifactor authentication
Keyboard remapping available in applications 
Media Player features need  keyboard access
Skip Navigation Links

Multifactor authentication.  Umar has set up a series of three-word passwords with a macro and uses a password manager for initial log-ins but if multifactor authentication is required he prefers to use finger recognition as long as he can keep his hand steady.  The problem is that biometrics are rarely used for website access so that is when Umar has to depend on his password manager and the Microsoft authenticator app to access secure sites such as his university login and for remote learning. (Web Content Accessibility Guidelines (WCAG) 2.2 Success Criterion 3.3.7 Accessible Authentication)

Keyboard Access is essential for Umar as he really finds using his joystick with the pointer accuracy required for some tasks difficult.  Single tabbing and using link lists and headings with macros all help with navigation but depend of the accessibility of the website.  He sometimes prefers to use his Xbox adaptive controller when feeling tired, as it offers switch access that can also be mapped to various access actions. Microsoft offer a list of keyboard shortcuts in apps

Tetralogic YouTube Video 1.02mins Watch this it is short and to the point! Quick accessibility test: Keyboard support

Keyboard remapping available in any application helps Umar as much as it helped Faith.  The WebAim team provide a very useful table that covers the standard keystrokes, but Umar tends to use his autohotkey personalised keyboard mapping

Skip Navigation Links  can be really helpful when wishing to get to the main part of the screen rather than having to tab all the way around and for Umar it is important that he sees this feature as he does not use a screen reader (Webaim guidance).  WCAG Success Criterion 2.4.1 Bypass Blocks  “Because navigating by headings or regions is not supported in most browsers, WebAIM recommends a “skip” link (in addition to headings and regions) to best support sighted keyboard users.”

Make dynamic content accessible this applies to everyone, but for Umar it is essential that all types of dynamic content react to keyboard use not just a mouse hovering over an item or a modal window that appears without activation or error messages that prevent an escape route.  IBM offer some potential Coding interactions – Dynamic content.   “Success Criterion 3.2.1 On Focus (Level A): When any user interface component receives focus, it does not initiate a change of context.

Headings for easy access.  This is not just about using consistent Heading order but being clear about titles and their meaning to help Faith quickly scan down content especially if there are important instructions.   This can help her concentration as well as make it easier to navigate.

Media player access.  Once again this is about using the keyboard to control:

  • play and pause buttons
  • the volume
  • screen size and exiting from full-screen and position
  • closed caption settings
  • caption and audio description settings
  • advance, rewind and timestamp
  • chat, share, embed and downloads
  • any other available player features and settings

Mozilla offer a guide to accessible multimedia

Key points from Umar

“I find small links, such as individual letters in an alphabet list really annoying when having to access a glossary if a search is not provided – too small to touch accurately or use with my joystick and too long for constant keystrokes. Either add white space around them or offer an alternative!”

Tetralogic YouTibe Video 3.15 – Browsing with a Keyboard YouTube

AbleGamers have produced a really helpful guide for developers about making ‘Accessible Player Experiences (APX)‘ that can be applied for any app development.

Webaim Motor Disabilities – Key Concepts for accessibility

IBM Coding Interactions – Keyboard interactions

Persona – Paul enjoys designing and finding technological solutions

Paul

Paul fair skin wiht brown hair

Age: 19 years

Course: Computer Science 1st Year

Hobbies: Rugby, travelling, films and social life

Background:

Paul really struggled at primary school and never enjoyed writing with endless corrections and not being very dextrous when it came to cursive script.  Eventually his difficulties with literacy skills were assessed and he received some extra help, but he never really talked about his dyslexia and was anxious about the mistakes he made, even though he knew his father had had similar difficulties and ran his own very successful IT company.  However, Paul soon discovered he was in a very different situation on the sports field and any lack of coordination seemed to disappear with the local youth rugby team giving him a chance to shine.  He not only captained the side during his sixth form years, but went on to join the university team in his first year and immediately found friends that allowed him to work around his lack of confidence, when it came to the academic side of things.

Paul not only realised there were others who had similar difficulties with course work and lecture notes.  He also requested extra time to complete written tasks and the use of a laptop as neat handwriting and getting ideas down on paper remained a challenge.  Choosing a science degree made it easier as it fitted with Paul’s strengths. When it came to problem solving, he found he could visualise the whole thing and often see a solution rather quicker than some other students, who took a linear path.  Now Paul openly admits that in his rush to get down his ideas he may make copying and spelling errors and his memory and organisational skills can let him down.  However, he has also learnt a range of strategies built up over time, such as downloading reminder, task and password apps onto his mobile phone.  He uses computer applications that auto-correct and read text aloud, plus looking out for supportive tools that highlight errors when compiling code and making use of graphical interfaces for system and software design.  Paul’s father is an IT systems analyst and computers have always been part of his life.  

Once Paul has a concept firmly in his mind, good analytical skills have meant that he can discuss possible algorithms or practical ways to get something to work.  He really enjoys projects and working with a group, as this allows him time to bring his abilities to the fore, whilst letting others do the report writing and documentation!  

Strategies to overcome Barriers to Access

Control over look and feel of a web service 
Keyboard Access
Multifactor authentication
Consistent Navigation and Heading order
Form Fields and Instructions
Error Identification
Selectable text for speech output 
Document accessibility

Multifactor authentication is a challenge especially when copying or remembering passwords is a necessity.  Paul uses the required Microsoft Authentication app for university logins but he largely depends on biometric logins (Android code) or secure second-factor authentication without SMS texts where possible (these are beginning to appear on the web services).  He understands the need for secure VPN requirements when working from home and has set up a password manager as advised by the National Cyber Security Office . He also has several rules to help filter spam emails on an Outlook account.   

When coding he has considered the Web Content Accessibility Guidelines (WCAG) 2.2 Success Criterion 3.3.7 Accessible Authentication as they have the potential to make developers aware of the issues for those who have different types of cognitive impairment including dyslexia.

Keyboard Access is important for Paul as he often uses keyboard shortcuts when writing code having learnt these over time.  He also finds it helpful when considering speech recognition whilst developing websites, as access and navigation can be made much easier (W3C WAI links for keyboard access related to Speech Recognition).

Consistent Navigation (WCAG) and Heading order (Webaim) helps Paul follow the flow of a web site. Clear, well-structured and easy to use menus that are consistent on every page are as much about usability as they are accessibility and the size of the headings with bold text can indicate the importance of content that needs to be read.

Form Fields and Instructions (WCAG) Guidance and clear indicators of what is needed in form fields can enhance completion rates and prevent errors.  Labels need to be near the edit areas (G162: Positioning labels to maximize predictability of relationships)

Error Identification (WCAG)  Any errors that are made should be flagged up with helpful correction notices that do not just depend on colour or cryptic messages.  The latter can often occur when a login is required and the user is none the wiser as to what is missing or incorrect.  At all times error messages must be able to be read aloud with text to speech and be informative.  Mozilla show how an ARIA alert role can be used when a ‘session is due to expire or connection to the server fails’. 

Selectable text for speech output – This is not quite the same as screen reading, where navigational elements as well as content are read aloud.  Paul likes to choose when he wants to highlight text and have it read out.  He can do this easily on his mobile phone with text messages, but finds on some websites and with certain PDFs that text may not be accessible.  Google calls this ‘select to speak’.  It is important to be wary about making text just available for screen readers.  You may present barriers to other users, so make sure visible text can highlighted by keyboard or mouse use when testing for accessibility. 

Control over look and feel of a web service    This can involve customisation as well as personalisation Paul usually adapts the user interface to suit his needs, as well as being able to select different settings in applications.  He sometimes wants to have an overlay to reduce the glare of black on white text and often changes font sizes and styles plus line spacing so these need to be adaptable.  He turns off any animated or timed elements and will stop blinking or scrolling text, as he finds this much harder to read.  He also does not like moving backgrounds or animated non-text elements that cause distractions.   An example of a site that is known for these issues is Lingscars . WCAG Success Criterion 2.2.2 Pause, Stop, Hide (Level A)

Document accessibility is important for everyone and when Paul uses his text to speech apps he expects to be able to have content read aloud at any speed in an order that makes sense, with a voice of his choice.  WebAim have an easy to follow set of instructions for making ‘Accessible Documents: Word, PowerPoint, & Acrobat’.

Key points from Paul

I like to understand what a web site or service is about immediately! If it is cluttered and I have to navigate through several layers to get to content I give up. Keep it simple is a good motto for me because I am impatient and don’t want to read long introductions.  Just give me a one liner and a diagram or picture that says it all!”

If you have a moment think about reading “Making Content Usable for People with Cognitive and Learning Disabilities” (W3C Editor’s Draft 21 September 2021) where the authors have provided a design guide that includes eight objectives namely:

  1. Help users understand what things are and how to use them
  2. Help users find what they need
  3. Use clear and understandable content
  4. Help users avoid mistakes and know how to correct them
  5. Help users focus
  6. Ensure processes do not rely on memory
  7. Provide help and support
  8. Support adaptation and personalization
Website accessibility testing – readability, colour contrast and dyslexia – Access iQ™ 4.51 mins (2012) YouTube https://youtu.be/TCCtaTz_4VU

Paul Crichton 2019 on UserZoom “Five ways to make your digital content easier to use and understand for people with dyslexia. ”How to make better websites for people with dyslexia

Professor Stein’s article on the BCS website, The Chartered Institute for IT  Why dyslexics make good coders

Persona – Faith keen to explore the use of virtual reality and accessibility

Faith

Age: 53 years

Course: Computer Science 3rd Year

Hobbies: Garden design, family and video games

Background:

Faith recently had COVID-19 and this really affected her arthritis.  She felt so tired and upset at times.  She was very upset that all her work was being affected by feeling as if she had brain fog, as well as having depleted levels of concentration.  On top of all the COVID challenges, she developed repetitive strain injuries (RSI) trying to cope with her studies as well as life in general.  Despite the fact that Faith’s family were now away from home much of the time, she was struggling but determined to finish her third-year project that involved access to virtual worlds with the development of three-dimensional (3D) Internet environments.   This was exciting, interesting but a challenge with a write up, as well as all the coding.

Having been an administrator, she had been bored with the endless spreadsheets and databases and decided to return to her love of mathematics and design rather, than looking at poorly developed user interfaces.  She wanted to be more creative and create ideas around virtual worlds that involved outdoor environments and in particular intricate garden designs, as this had become more than just a hobby.   However, as her arthritis progressed and now her RSI was really affecting her dexterity, she was concerned that she would not be able to cope with so much mouse work, such as click and hold to grab a screen or multiple clicks to drag and drop items. 

Taking breaks had become essential and accessing controls via keyboard shortcuts seemed to help, but the latter was not always possible when creating demonstration applications.  Faith found she needed to use her left hand more and keep her right forearm in the handshake position to prevent the RSI pain.   Using an upright mouse with a roller ball on the side or a pen stylus solved some issues.  She really liked to be able to step through her worlds using single keys to move around and had her Mac set up with several personalised keyboard shortcuts using Control accessibility options with her keyboard.  Speech recognition was good for much of Faith’s writing, when she used the dictation settings and Voice Control for the main navigation challenges on the computer.  However, remembering all the commands was hard, so Faith resorted to a crib sheet!

Main Strategies to overcome Barriers to Access

Control and navigation labelling for speech recognition. 
Keyboard Access for navigation
Multifactor authentication
Keyboard remapping available in applications 
Headings for easy access
Skip Navigation Links

Strategies to overcome Barriers to Access

Multifactor Authentication. Faith finds fiddling with two devices really difficult especially when she is then required to copy a code or manage a series of numbers. She much prefers to use biometric forms of verification, such as speech, face or finger recognition; her computer and phone have her preferred finger recognition. At the moment she cannot remember passwords or phrases let alone three random words. Faith has to cope with the Microsoft authenticator app to access university sites and has a password manager system for online access. (Web Content Accessibility Guidelines (WCAG) 2.2 Success Criterion 3.3.7 Accessible Authentication)

Keyboard Access  “Speech recognition can be used for dictating text in a form field, as well as navigating to and activating links, buttons, and other controls.” (W3C WAI speech recognition accessibility) The link to making this possible is to ensure that all these controls are keyboard accessible and then the mapping of the commands will work with voice controls.    Example of how other criteria linked to WCAG compliance help solve the challenges faced by Faith

Keyboard remapping available in applications in order to personalise keyboard controls and make it easier to access app components.  The WebAim team provide a very useful table that covers the standard keystrokes that Faith uses for the browser interactions, with hints that enhance accessibility.  

Control and navigation labelling for speech recognition.  The way controls are labelled can make all the difference when accessing sites using a keyboard or speech.  Faith finds she has to use her spoken command crib sheet more and more to access controls and to navigate around pages if she cannot use her mouse and these commands depend on the labels that have been used by the developer.

Skip Navigation Links  can be really helpful when wishing to get to the main part of the screen rather than having to tab all the way around and for Faith it is important that she sees this feature as she does not use a screen reader (Webaim guidance).  WCAG Success Criterion 2.4.1 Bypass Blocks  “Because navigating by headings or regions is not supported in most browsers, WebAIM recommends a “skip” link (in addition to headings and regions) to best support sighted keyboard users.”

Headings for easy access.  This is not just about using consistent Heading order but being clear about titles and their meaning to help Faith quickly scan down content especially if there are important instructions.   This can help her concentration as well as make it easier to navigate.


Key points from Faith

“The worse thing is that I have to take regular breaks now because my concentration goes and I know if I spend more than half an hour continuously using my mouse I will be in pain for the rest of the day. I really want technology applications that are used for designing and developing apps to be more accessible as well as the apps themselves”

AbleGamers have produced a really helpful guide for developers about making ‘Accessible Player Experiences (APX)‘ that can be applied for any app development.

Speech recognition on Mac YouTube Using Voice Control To Click and Drag With Speech Commands

Accessibility Considerations for Augmented and Virtual Reality for the Classroom and Beyond from the Bureau of Internet Accessibility (USA)