EVENT SCALE (Twitter virality and prominence scale)
Measuring event mediatisation on Twitter
With Twitter’s positioning as a key social medium for public debate, qualitative and quantitative analysis of messages posted on it has become an integral part of media monitoring. However, basic data such analysis produces – numbers of tweets and occurrences of hashtags, etc. – is not directly or intuitively comprehensible to non-experts (unlike television audience ratings, for example).
This being so, the Government Information Department (SIG), one of whose main concerns is media monitoring, wanted to provide itself with a tool enabling construction of a memory of events on Twitter, along with their classification according to relative intensity and association with a clear reference system that most people would have no trouble understanding.
This is why the EVENT open scale was developed; it measures an event’s “explosiveness” on Twitter at the time it produces the most tweets. Each rung of the scale corresponds to a quantity of tweets and is linked to a newsworthy event, providing a more accurate and intuitive idea of how any given rung expresses itself in terms of “buzz” intensity.
The Government Information Department will henceforth use the EVENT scale to qualify and standardise the Twitter tracking reports it produces and disseminates.
EVENT was designed as a tool for comprehension and appropriation, initially but by no means exclusively for use in government circles; we are presenting the reasoning and methodology behind it here so that others can understand it. It does not claim to express a scientific truth or avoid all the possible biases that such types of analysis can incorporate, but rather to provide operational and empirical intelligence on an event’s “Twittesque” intensity.
How the EVENT scale was developed
After spending a year creating a Twitter event database, we observed that different items often came together to form a single group and noted quantitative aggregations at several levels according to event typology (tragedies, natural disasters, sporting events, celebrity deaths, etc.). We decided to select these levels of “natural” aggregation to serve as rungs on the scale, resulting in our providing it with 9 levels in all, each constituting a tier of intensity between 10,000 (level 1) and 6.5 million or more tweets (level 9).
In order to include a media sequence identified as being of political interest in the database:
- The first step is to define a query enabling measurement of the number of tweets concerning it. The query is made up of keywords (hashtags, expressions, name of a political figure or reform, etc.). Although its construction is empirical by nature and would be unlikely to cover all tweets concerned or avoid capturing “noise”, continued use of the same composition principles ensures relative accuracy of query results in relation to one another. Measurement is currently carried out via the Visibrain “Quick Trends” tool, which records the volume of tweets corresponding to the query, with access to all Twitter messages (Firehose).
- It is then a matter of defining the most appropriate time period (e.g. one day for an interview, a few days for a natural disaster, or several weeks for such overwhelming events as terrorist attacks). When an event takes place over time (such as the refugee crisis in Calais), it may be best to measure it over weekly time periods, from Monday to Sunday inclusive, in order to carry out long-term monitoring. Bear in mind that the list of events is by no means exhaustive, in particular as regards political interviews.
The database therefore includes:
- The overall volume of tweets over the selected period, in order to make an overall assessment of the phenomenon. The minimum unit is one full day.
- The volume of tweets over the day during which the phenomenon was most visible, in order to measure the intensity of activity on a given day. To obtain this, the day with the most tweets is identified and the number of tweets on that day is divided by 12 (conventional factor corresponding to the average overall number of hours of activity on Twitter per day). This gives an average number of tweets per hour on the day when mediatisation of the event in question reached its peak. The number of tweets over the highest peak hour is also available, in order to measure record intensity.
- In cases of political interviews, the overall volume over the 4 hours following the interview.
Another part of the file is devoted to online mobilisations. The aim here is to measure the exact volume of tweets around one or more specific hashtags, sometimes used on a massive scale by Twitterites looking to defend or denounce a cause, person or measure (#RéformeOrthographe), or even pay tribute (#PrayForParis). Data collection procedures are the same as for the first page: composition of query, definition of time period, indication of overall volume of tweets over the selected period, along with the volume of tweets over the peak day, the average number of tweets per hour on the peak day and the volume of tweets over the peak hour of the peak day.
How are Twitter event database queries composed?
The database that the EVENT scale draws on is more of a relative indicator than a tool for exact scientific measurement. Queries cannot claim to capture all tweets devoted to a given topic, due in particular to possible Twitter-user errors in hashtags or residual omissions. Conversely, “false positives” are also possible. Nonetheless, volumes obtained are representative of flows and exchanges between Twitterites and provide indications of orders of magnitude.
Queries are composed in a series of steps:
- Listing of the main hashtags making the rounds,
- Observation of angles and expressions used by the media and Twitter users on the topic in question,
- Organisation of possible responses to questions on “who?” (protagonists), “what?” (statements or events) and “where?” (places where the event occurred),
- Inputting the query in Visibrain “Quick Trends”, monitoring software with reliable and exhaustive access to Twitter’s Firehose. Constraints: limited query size (30 words) and precedence (limited to 30 days).
- French language tweets only (systematic use of “lang:fr”),
- First of all, list the most used hashtags specific to the event and within the timeframe,
- Then create more detailed queries organised into “who, what and where”,
- Limit and identify off-subject “noise” in order to refine the query,
- Select the most appropriate measurement period (start date, end date and peak day).
Example of a query:
Event: PM’s announcement regarding young people
Specific hashtags: (PrioritéJeunesse OR PrioriteJeunesse) lang:fr
Who/Where: (Valls OR ManuelValls OR gouvernement OR gouvernementFR OR Matignon) What: (jeunes OR jeunesse OR organisation OR organisations OR syndicat OR syndicats OR unef OR fage OR lycéens OR étudiants OR lycéennes OR étudiantes) lang:fr