Measuring user satisfaction
Indicators
The following indicators are used in the Publications Office to measure user satisfaction.
| Indicator | Description |
|---|---|
| Publications Office Portal user satisfaction rate | Indicates the quality of the service provided by the Publications Office Portal to its various customers. Shown through the percentage of neutral and positive opinions expressed in the portal’s user survey. Registered and ‘privileged’ users, users who have recently ordered publications and author services are targeted, among others. |
| CORDIS user satisfaction rate | Percentage of neutral and positive opinions expressed in the annual user survey. |
| TED | Percentage of neutral and positive opinions expressed in the user surveys regularly conducted on TED. |
Surveys
Measuring user satisfaction is of great importance to any future improvements in the Publications Office’s web presence. A well-planned survey provides direct user feedback and relevant information on the user journey, expectations and pain points. Surveys provide valuable insight into the real experience of citizens interacting with the Publications Office’s digital services and content.
How it works
Depending on the type of information required, the scale of the research needed and the nature of the planned improvement, there are two types of potential surveys that can be carried out.
If a general in-depth study of the user and global platform features is required, a long-format survey is more suitable. On the other hand, if a specific problem or issue is to be improved, a planned new feature is under discussion or a quick status check of the platform’s performance is needed, the short-format survey should be used instead.
Long-format survey
Long-format survey
This type of survey is thorough. It includes a list of questions and provides more information about the users and their overall experience. Ideally, the survey should have its own page due to the multiple questions it contains; it should not be placed in a scrollable pop-up which would make it hard to fill in, for example.
Methodology
It is essential to decide what to measure, how to measure it and who to survey. Define the survey’s time span and sample size, i.e. how long it should run for and how many respondents will be needed to get a sufficient number of responses for a conclusion.
Analysing the results
If the initial survey planning is clear, results should be easy to read. A given hypothesis will or will not match with the user responses provided. For example, if gathering insights on new and infrequent users, answers should be filtered by website visit frequency. If feedback on advanced features is needed, a specific question filtering for knowledgeable users should be taken into account.
Survey script
If you need help and insight on drafting your survey questions, please consult:
https://wikis.ec.europa.eu/display/WEBGUIDE/02.+Measure+user+experience%3A+the+Satisfaction+Index#id-02.Measureuserexperience:theSatisfactionIndex-Surveyscript
Short-format survey (polling)
About the format
This type of survey is short and very specific. It provides an answer to a particular question or problem. Usually, short-format surveys (polling) contain only one question, which makes them appealing and easy for users to complete.
Methodology
Due to the specifics of the survey, only one question should be posed. The question should have either a binary (yes/no) answer or a maximum of four possible options (such as yes / no / not sure or options 1/2/3/4).
The question should be phrased in a way which avoids bias (i.e. it should not influence the user’s answer) and should ideally be short and easy to comprehend. The survey’s time span and sample size should be defined.
Analysing the results
This type of survey measures who the majority of users are and what percentage of users have chosen a particular answer to a question. As an example, let’s imagine a month-long survey with the question ‘How often do you visit the Publications Office website?’ The possible answers are ‘daily’, ‘weekly’, ‘monthly’, ‘once a year’ and ‘it’s my first time’. The predefined user types are ‘frequent’, ‘moderate’ and ‘new’. Based on the answers, we can establish which of these three types constitutes the majority of users, which comes second and which comes third. The survey can be executed twice a year to validate results and avoid bias.
How to request this service
In-house tools – long-format surveys
If you would like to run your survey internally, send an email to op-survey@publications.europa.eu with a description of your project and we’ll be glad to guide you through the process.
More information is available here (EU Login required).
Another option is to create your survey using EUSurvey (https://ec.europa.eu/eusurvey/home/welcome ).
Get help from a contractor
Please contact Ludovic Foigne (ludovic.foigne@publications.europa.eu ).