H2020 Mobi-QoE

Project: H2020 Mobi-QoE

Mobi-QoE (Monitoring and Analysis of Quality of Experience in Mobile Broadband Networks, funded by European Commission H2020 project MONROE (Measuring Mobile Broadband Networks in Europe))

The objective of Mobi-QoE is to extend MONROE’s testbed to the QoE domain by integrating novel software-based QoE-capable measurement tools and QoE models for popular end-user services (e.g., YouTube, Facebook, Spotify). In crowdsourced field trials, the extensions will be evaluated and the QoE models will be refined to enable experiments for QoE-based performance analysis of mobile broadband networks with the MONROE testbed.

The project Mobi-QoE is conducted in collaboration with AIT Austrian Institute of Technology GmbH

Researchers:

Dr. Florian Wamser,
Anika Schwind M.Sc.

Pedro Casas,
Dr. Michael Seufert

Tool: YoMo-Docker

Yomo-Docker is a Docker container to actively measure QoE related factors of YouTube video streaming. The measurement concept is based on emulating a virtual end-user device requesting video streams, which are then monitored at the network and application layers, on the basis of QoE-relevant features.

It is available on DockerHub by pulling mobiqoe/yomo_docker or on GitHub in the yomo-docker repository.

For more information, please contact Anika Schwind (anika.schwind@informatik.uni-wuerzburg.de).

Local Testing

To start local QoE tests, Docker has to be installed. Then, simply run the following command and get the results into a selected folder:

docker run --cap-add=NET_ADMIN --env LOCAL=1 -v <Path to result folder>:/monroe/results yomo_docker

or use a config file in addition to specify YouTube ID, duration and bitrates for different quality levels:

docker run --cap-add=NET_ADMIN --env LOCAL=1 -v <Path to config file>:/monroe/config -v <Path to result folder>:/monroe/results yomo_docker

Output

The container will export three different log files:

Information about the playout buffer (in intervals of approx. 1s) in the following format:
```
	timestamp#video playback time#buffered playback time#available playback time\n 
```
Information about the video player (playback events, video information) in the following format:
```
	timestamp#information\n 
```

Statistics about bitrate, buffer, and stallings

	avg, max, min, 25-50-75-90 quantiles of: bitrate [KB], buffer [s], number of stalls (only one value), duration of stalls

Network traffic information during the video playback using tshark

Publications

2018[ to top ]

Streaming Characteristics of Spotify Sessions. Schwind, Anika; Wamser, Florian; Gensler, Thomas; Seufert, Michael; Casas, Pedro; Tran-Gia, Phuoc. In The 2nd International Workshop on Quality of Experience Management. Sardinia, Italy, 2018.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Internet Service Providers need a thorough understanding of a service to maximize the Quality of Experience (QoE) of their customers by network management. Instead of quantifying the user satisfaction with long and cost-intensive subjective user studies, the QoE can often be estimated with the help of dedicated measurements of application and network parameters. We designed a QoE measurement tool for the popular audio streaming service Spotify that runs inside a Docker software container. The container is able to run headlessly as active measurement probe and emulates a user who is streaming audio files via Spotify. While streaming, network and application parameters are collected that have a high correlation to the user's QoE. The results of the measurements are used to characterize audio streaming in Spotify on application and network layer, and to evaluate important QoE factors.

@inproceedings{info3-inproceedings-2018-13, abstract = {Internet Service Providers need a thorough understanding of a service to maximize the Quality of Experience (QoE) of their customers by network management. Instead of quantifying the user satisfaction with long and cost-intensive subjective user studies, the QoE can often be estimated with the help of dedicated measurements of application and network parameters. We designed a QoE measurement tool for the popular audio streaming service Spotify that runs inside a Docker software container. The container is able to run headlessly as active measurement probe and emulates a user who is streaming audio files via Spotify. While streaming, network and application parameters are collected that have a high correlation to the user's QoE. The results of the measurements are used to characterize audio streaming in Spotify on application and network layer, and to evaluate important QoE factors.}, address = {Sardinia, Italy}, author = {Schwind, Anika and Wamser, Florian and Gensler, Thomas and Seufert, Michael and Casas, Pedro and Tran-Gia, Phuoc}, booktitle = {The 2nd International Workshop on Quality of Experience Management}, keywords = {can}, month = {05}, title = {Streaming Characteristics of Spotify Sessions}, year = 2018 }

%0 Conference Paper %1 info3-inproceedings-2018-13 %A Schwind, Anika %A Wamser, Florian %A Gensler, Thomas %A Seufert, Michael %A Casas, Pedro %A Tran-Gia, Phuoc %B The 2nd International Workshop on Quality of Experience Management %C Sardinia, Italy %D 2018 %T Streaming Characteristics of Spotify Sessions %X Internet Service Providers need a thorough understanding of a service to maximize the Quality of Experience (QoE) of their customers by network management. Instead of quantifying the user satisfaction with long and cost-intensive subjective user studies, the QoE can often be estimated with the help of dedicated measurements of application and network parameters. We designed a QoE measurement tool for the popular audio streaming service Spotify that runs inside a Docker software container. The container is able to run headlessly as active measurement probe and emulates a user who is streaming audio files via Spotify. While streaming, network and application parameters are collected that have a high correlation to the user's QoE. The results of the measurements are used to characterize audio streaming in Spotify on application and network layer, and to evaluate important QoE factors.

2017[ to top ]

YouTube QoE Monitoring with YoMoApp: a Mobile App for Crowdsourced YouTube QoE Analysis. Seufert, Michael; Wehner, Nikolas; Wamser, Florian; Casas, Pedro. London, UK, 2017, November.
- [ BibTeX ]
- [ EndNote ]
@misc{info3-poster-2017-1, address = {London, UK}, author = {Seufert, Michael and Wehner, Nikolas and Wamser, Florian and Casas, Pedro}, booktitle = {ACM Internet Measurement Conference (IMC)}, keywords = {can}, month = 11, title = {YouTube QoE Monitoring with YoMoApp: a Mobile App for Crowdsourced YouTube QoE Analysis}, year = 2017 }

%0 Generic %1 info3-poster-2017-1 %A Seufert, Michael %A Wehner, Nikolas %A Wamser, Florian %A Casas, Pedro %B ACM Internet Measurement Conference (IMC) %C London, UK %D 2017 %T YouTube QoE Monitoring with YoMoApp: a Mobile App for Crowdsourced YouTube QoE Analysis
Concept and Implementation of Video QoE Measurements in a Mobile Broadband Testbed. Schwind, Anika; Seufert, Michael; Alay, Özgü; Casas, Pedro; Tran-Gia, Phuoc; Wamser, Florian. In IEEE/IFIP Workshop on Mobile Network Measurement (MNM’17). Dublin, Ireland, 2017.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
The MONROE testbed enables the objective performance assessment of MBB networks from the end-user perspective, using highly distributed measurements from fixed and mobile nodes. To quantify the performance of MBB networks for popular Internet services from a user-centric perspective, dedicated tools are needed. In this paper we extend the MONROE testbed to the Quality of Experience (QoE) domain, presenting the design and implementation of a QoE-capable measurement tool for YouTube video streaming. The measurement concept is based on emulating a virtual end-user device requesting video streams, which are then monitored at the network and application layers, on the basis of QoE-relevant features. The initial measurements conducted in the MONROE testbed and reported in this paper demonstrate the applicability of the implemented measurement concept.

@inproceedings{info3-inproceedings-2017-15, abstract = {The MONROE testbed enables the objective performance assessment of MBB networks from the end-user perspective, using highly distributed measurements from fixed and mobile nodes. To quantify the performance of MBB networks for popular Internet services from a user-centric perspective, dedicated tools are needed. In this paper we extend the MONROE testbed to the Quality of Experience (QoE) domain, presenting the design and implementation of a QoE-capable measurement tool for YouTube video streaming. The measurement concept is based on emulating a virtual end-user device requesting video streams, which are then monitored at the network and application layers, on the basis of QoE-relevant features. The initial measurements conducted in the MONROE testbed and reported in this paper demonstrate the applicability of the implemented measurement concept.}, address = {Dublin, Ireland}, author = {Schwind, Anika and Seufert, Michael and Alay, Özgü and Casas, Pedro and Tran-Gia, Phuoc and Wamser, Florian}, booktitle = {IEEE/IFIP Workshop on Mobile Network Measurement (MNM’17)}, keywords = {can}, month = {06}, title = {Concept and Implementation of Video QoE Measurements in a Mobile Broadband Testbed}, year = 2017 }

%0 Conference Paper %1 info3-inproceedings-2017-15 %A Schwind, Anika %A Seufert, Michael %A Alay, Özgü %A Casas, Pedro %A Tran-Gia, Phuoc %A Wamser, Florian %B IEEE/IFIP Workshop on Mobile Network Measurement (MNM’17) %C Dublin, Ireland %D 2017 %T Concept and Implementation of Video QoE Measurements in a Mobile Broadband Testbed %X The MONROE testbed enables the objective performance assessment of MBB networks from the end-user perspective, using highly distributed measurements from fixed and mobile nodes. To quantify the performance of MBB networks for popular Internet services from a user-centric perspective, dedicated tools are needed. In this paper we extend the MONROE testbed to the Quality of Experience (QoE) domain, presenting the design and implementation of a QoE-capable measurement tool for YouTube video streaming. The measurement concept is based on emulating a virtual end-user device requesting video streams, which are then monitored at the network and application layers, on the basis of QoE-relevant features. The initial measurements conducted in the MONROE testbed and reported in this paper demonstrate the applicability of the implemented measurement concept.
Unsupervised QoE Field Study for Mobile YouTube Video Streaming with YoMoApp. Seufert, Michael; Wehner, Nikolas; Wamser, Florian; Casas, Pedro; D’Alconzo, Alessandro; Tran-Gia, Phuoc. In 9th International Conference on Quality of Multimedia Experience (QoMEX). Erfurt, Germany, 2017.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
YoMoApp (YouTube Monitoring App) is an Android app to monitor mobile YouTube video streaming on both application- and network-layer. Additionally, it allows to collect subjective Quality of Experience (QoE) feedback of end users. During the development of the app, the stable versions of YoMoApp were already available in the Google Play Store, and the app was downloaded, installed, and used on many devices to monitor streaming sessions. As the app was not advertised in special campaigns or used for dedicated QoE studies, the monitored streaming sessions of this period compose the data set of a large unsupervised field study. The collected data set is evaluated to characterize current mobile YouTube streaming on both application and network layers. Furthermore, the problems and methodology to obtain QoE results from such unsupervised field study are discussed together with the actual QoE results. Correlations between QoE factors are investigated, and the QoE of clusters of similar streaming sessions is analyzed.

@inproceedings{info3-inproceedings-2017-9, abstract = {YoMoApp (YouTube Monitoring App) is an Android app to monitor mobile YouTube video streaming on both application- and network-layer. Additionally, it allows to collect subjective Quality of Experience (QoE) feedback of end users. During the development of the app, the stable versions of YoMoApp were already available in the Google Play Store, and the app was downloaded, installed, and used on many devices to monitor streaming sessions. As the app was not advertised in special campaigns or used for dedicated QoE studies, the monitored streaming sessions of this period compose the data set of a large unsupervised field study. The collected data set is evaluated to characterize current mobile YouTube streaming on both application and network layers. Furthermore, the problems and methodology to obtain QoE results from such unsupervised field study are discussed together with the actual QoE results. Correlations between QoE factors are investigated, and the QoE of clusters of similar streaming sessions is analyzed.}, address = {Erfurt, Germany}, author = {Seufert, Michael and Wehner, Nikolas and Wamser, Florian and Casas, Pedro and D'Alconzo, Alessandro and Tran-Gia, Phuoc}, booktitle = {9th International Conference on Quality of Multimedia Experience (QoMEX)}, keywords = {can}, month = {05}, title = {Unsupervised QoE Field Study for Mobile YouTube Video Streaming with YoMoApp}, year = 2017 }

%0 Conference Paper %1 info3-inproceedings-2017-9 %A Seufert, Michael %A Wehner, Nikolas %A Wamser, Florian %A Casas, Pedro %A D'Alconzo, Alessandro %A Tran-Gia, Phuoc %B 9th International Conference on Quality of Multimedia Experience (QoMEX) %C Erfurt, Germany %D 2017 %T Unsupervised QoE Field Study for Mobile YouTube Video Streaming with YoMoApp %X YoMoApp (YouTube Monitoring App) is an Android app to monitor mobile YouTube video streaming on both application- and network-layer. Additionally, it allows to collect subjective Quality of Experience (QoE) feedback of end users. During the development of the app, the stable versions of YoMoApp were already available in the Google Play Store, and the app was downloaded, installed, and used on many devices to monitor streaming sessions. As the app was not advertised in special campaigns or used for dedicated QoE studies, the monitored streaming sessions of this period compose the data set of a large unsupervised field study. The collected data set is evaluated to characterize current mobile YouTube streaming on both application and network layers. Furthermore, the problems and methodology to obtain QoE results from such unsupervised field study are discussed together with the actual QoE results. Correlations between QoE factors are investigated, and the QoE of clusters of similar streaming sessions is analyzed.
Unperturbed Video Streaming QoE Under Web Page Related Context Factors. Seufert, Michael; Zach, Ondrej; Slanina, Martin; Tran-Gia, Phuoc. In 9th International Conference on Quality of Multimedia Experience (QoMEX). Erfurt, Germany, 2017.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Quality of Experience (QoE) of Internet services is affected by human, system, and context influence factors. While most QoE studies so far are focused on system factors only, this work will assess the impact of context factors of video streaming on QoE. As video streaming is mostly consumed from web pages, such as video portals, the investigated test conditions are applied to the web page, which embeds the video player. Therefore, the study of context factors is implicitly conducted within a crowdsourced QoE study. The test conditions considered different page load times, poster image qualities, and displayed advertisements on the web page, which are typical context factors when consuming a video streaming service. The results of the study show that the modification of the context factors on the streaming web page leaves the users' QoE rating unperturbed, which suggests that the investigated context factors have a negligible impact on video streaming QoE, or that the rating task of the subjective QoE study superimposed the context factors.

@inproceedings{info3-inproceedings-2017-11, abstract = {Quality of Experience (QoE) of Internet services is affected by human, system, and context influence factors. While most QoE studies so far are focused on system factors only, this work will assess the impact of context factors of video streaming on QoE. As video streaming is mostly consumed from web pages, such as video portals, the investigated test conditions are applied to the web page, which embeds the video player. Therefore, the study of context factors is implicitly conducted within a crowdsourced QoE study. The test conditions considered different page load times, poster image qualities, and displayed advertisements on the web page, which are typical context factors when consuming a video streaming service. The results of the study show that the modification of the context factors on the streaming web page leaves the users' QoE rating unperturbed, which suggests that the investigated context factors have a negligible impact on video streaming QoE, or that the rating task of the subjective QoE study superimposed the context factors.}, address = {Erfurt, Germany}, author = {Seufert, Michael and Zach, Ondrej and Slanina, Martin and Tran-Gia, Phuoc}, booktitle = {9th International Conference on Quality of Multimedia Experience (QoMEX)}, keywords = {can}, month = {05}, title = {Unperturbed Video Streaming QoE Under Web Page Related Context Factors}, year = 2017 }

%0 Conference Paper %1 info3-inproceedings-2017-11 %A Seufert, Michael %A Zach, Ondrej %A Slanina, Martin %A Tran-Gia, Phuoc %B 9th International Conference on Quality of Multimedia Experience (QoMEX) %C Erfurt, Germany %D 2017 %T Unperturbed Video Streaming QoE Under Web Page Related Context Factors %X Quality of Experience (QoE) of Internet services is affected by human, system, and context influence factors. While most QoE studies so far are focused on system factors only, this work will assess the impact of context factors of video streaming on QoE. As video streaming is mostly consumed from web pages, such as video portals, the investigated test conditions are applied to the web page, which embeds the video player. Therefore, the study of context factors is implicitly conducted within a crowdsourced QoE study. The test conditions considered different page load times, poster image qualities, and displayed advertisements on the web page, which are typical context factors when consuming a video streaming service. The results of the study show that the modification of the context factors on the streaming web page leaves the users' QoE rating unperturbed, which suggests that the investigated context factors have a negligible impact on video streaming QoE, or that the rating task of the subjective QoE study superimposed the context factors.

Student Works and Theses