Skip to main content

Table 2 Usability issues from prototype testing and focus group participants and recommended revisions

From: Development and usability testing of a patient decision aid for newly diagnosed relapsing multiple sclerosis patients

Area

Usability Issue

Recommended changes

Clarity

“Informative and very technical. Make it easier to understand – patient friendly.”

Simplify language.

“… [M]ore consistent separation between steps to inform versus decision making.”

Limit the amount of information on each page and present one question per page.

“[A]fter selecting from this page “ABOUT ME”, I thought “OPTIONS” were the selection of medications made for me. Especially because it personalizes the top of the page then continues on with Medications, so I assumed these were choices made for me. Eventually, of course, I realized “MY CHOICE” was the medication choices that were recommended for me.”

Revise ordering of information. Show information about treatment options in the “Introduction”, ask “About Me” questions, and then show best matches in “My Choice”.

“I found it confusing [referring to ‘doctor recommended’ treatment label]. I was like, what is best match over doctor recommended?”

Remove doctor recommended label for treatment options.

Content

“I’m not sure if I could clearly show what my symptoms have been since I was diagnosed.”

Add an open-ended question that allows patients to share their most concerning symptoms with the doctor.

“I would have liked to have more options to choose from for some of the questions.”

Add more response options to questions.

“It is important to me to know how long a drug had been on the market. I wanted to know the safety of the drug had been tested over a long period of time and that was part of my decision process.”

Add Health Canada approval date for each medication.

“And this is something that I’ve only ever seen talked about in one forum, like one of those physician forums. What is the actual impact of the treatment on the quality of life?”

Where possible, add information on the impact of treatment on quality of life to presentation of treatment options.

“I feel like being quizzed [all participants agree] on how well you read everything and memorized it.

“I think if you’re developing cognitive challenges, that would be a bit complicated.”

Revise quiz to be less burdensome.

Content Presentation

“Maybe have a grid of all medications laying out all relative details so that they can all be compared side by side instead of using the boxes that dropped down to read about them.” (5 patients)

Create a 1-page comparison chart that patients can access and save for their own use.

“[A] more detailed summary of the treatment options selected at the end.”

“Being able to consolidate the information with my doctor or other healthcare professionals would allow for optimal understanding and learning”

Improve summary page to contain detail patients want to see, and questions they might have for their doctor.

“I have to admit, the colour is an issue. …It definitely was a bit of a challenge with the bars and the colour contrasts…For me, if there’s not a strong enough contrast, it’s quite difficult to see.”

Improve contrast in information presentation. Darken background colour for information headings.

Video Content

“I didn’t think it [the video] added anything to it to it. It was fine, but I don’t know if it was necessary.”

Remove video describing MS.

“Well it [personal accounts] seems like a real thing as opposed to just the information. Like someone in your situation actually is saying that.”

“I think a testimonial would be a lot different than a factual video. I think it would be helpful to hear… And how it affected them, and how they felt about it.”

Include patient accounts of receiving an MS diagnosis, and the experience of choosing a treatment.

Links

“Good content. I liked the links to sources and studies, and found myself clicking to those links to find out more info. I would have liked to see a few more. Example, when selecting Goals, option “slow disease”, I would have liked to see a link for this “and the number of lesions on the first MRI may predict the level of impairment in the long run””

Add a page in the introduction to clarify source of information provided. Identify sections where additional source links can be added.

Functionality

“I didn’t like how it asked if I had taken a medication but had no option to say that I had adverse reaction to that medication. Later, it recommended that medication to me.”

Improve past medication history question – separate previous and current medication, and add question to clarify reason for stopping the medication.

Value Elicitation Method

“I think it would have been easier to think about the importance of goals based on a number scale rather than a slider that didn’t have a reference point.” (2 patients)

“[T]he sliders were a bit confusing. If those were my concerns it is likely I won’t want to lower the value of how I feel about what I want and don’t want. I felt the sliders were not needed.” (4 patients)

“I think [this] section could be expanded so there are some different options. It didn’t seem to capture individual differences as much as I would have liked.”

Refine value elicitation technique and add more explicit instructions.

“[I]nstead of having fixed statements maybe have somewhere to type what is important for each person in case their ideas were not covered in the statements to choose from.”

Add question in “Review” that asks if patient has any other goals they wish their doctor to know that was not addressed in the decision aid.

“I have something just not important at all. Just check a box or uncheck a box so it disappears from it.”

“Or like a not important, or unsure, don’t know.”

Include an option to rate some values as “unimportant” or “don’t know”.

Summary

“There’s no question in any of this that has to do with quality of life. And I wonder if, because somebody could be absolutely having worse time, but they think their quality of life is okay. And somebody, having minor symptoms and they think their quality of life is very poor. And I think that might be an interesting piece of information for the neurologist.”

Consider including a quality of life instrument.

“You’re self-reporting and how accurate are you?… you’re so scared in the beginning and I don’t know how beneficial that is [referring to comparing self-reported disability to population average]. I don’t know what other people think about that.”

“It is a little scary/overwhelming.”

Remove comparison to other MS patients.