‍

Context

‍

The UX team at FranceInfo has just completed work on the latest version of its mobile application. FranceInfo's Marketing Department wishes to assess the alignment between the application and certain strategic objectives of their online activity, particularly the satisfaction of their current users and their ability to attract new users.

They intend to conduct face-to-face user tests that can provide insights and improvement suggestions for the next update of the application. Additionally, they want to verify the performance of one of  their strategic customer journeys from a user experience perspective, compared to their main competitor, BBC News.

‍

#Elements requiring validation from user testing:

• the smoothness of navigation and access to the main sections of the application
• the ease of searching for and selecting an article, compared with BBC News
• discovering the offerings and personalization
• the comprehension and readability of the article pages
• the application's ability to bring users back

‍

‍

‍

‍

Team

‍

‍

‍

‍

‍

My role

‍

I was responsible for the overall test management in terms of methodology.

As a test strategist, my responsibilities included:

• Defining the test strategy and processes
• Drafting the test protocol
• Monitoring test sessions
• Analysing qualitative and quantitative data

‍

‍

‍

‍

Methodology

‍

‍

‍

‍

‍

Key data and findings

‍

#Participant profiles

For the testing process, our primary focus was on young adults, a demographic of particular interest for surveying as it offers insights into their habits regarding information retrieval and consumption in the era of digital social networks. Additionally, as a traditional media outlet, Franceinfo developed this application to appeal to a wide audience, making it crucial to understand what factors might encourage regular usage among a younger demographic.

‍

‍

‍# How we analyze

For each test, we posed 3 questions to the participant following each scenario or task in order to gather their feedback. Given our limited number of participants (14), we acknowledge that the data could sometimes be misleading and insufficient to extrapolate to the wider population. Therefore, we calculated 95% confidence intervals to provide a probabilistic estimate of how well a metric obtained from our study explains the behaviour of the entire user population.

Here is an example regarding the completion rate of each scenario. Additionally, we calculated the task time and satisfaction rate.

Please refer to the analysis section of our report to view all the results.

‍

‍

‍# Findings of key scenarios

For every scenario and task, we initially analysed the metrics and then reviewed participants' feedback gathered during the test session to pinpoint issues. Then we delved into the specifics of each problem to understand the target user's behaviour and preferences and explore potential opportunities to improve the feature.

Below is an example of the navigation and search test along with recommendations for addressing each identified pain point.

To view all the results, please refer to the first part of the analysis section in our report.

Below is another test scenario aimed at comparing the thematic navigation mode between Franceinfo and BBC News.

The goal is to evaluate the competitor’s designs and translate that evaluation into recommendations for Franceinfo. Following this, we have proposed an A/B test strategy with a hypothesis focusing on one variable: the "thematic label" on an article. Running this test will enable us to delve deeper into how users interact with each version and provide additional insights into the strengths and weaknesses of the redesign work.

‍

‍

‍# Findings of post-test interviews

After participants completed the tasks, we asked them a series of summative questions about their experiences using the Franceinfo application, considering aspects of design, usability, and satisfaction.

Here are some findings from our System Usability Scale (SUS) session.

To view all the results, please refer to the second and the third section of the analysis part in our report.

‍

‍

Insights

In the end, with all the data collected from our test sessions, we prioritised the insights and then transformed them into feasible design decisions. Although not as detailed as technical design instructions, these will serve for the Franceinfo project team as a first-level backlog to generate the improvement iteration work in the future.

‍

→To access our test report (only available in French at the moment)

‍

‍

‍

‍

‍

‍

_‍