SPI Introduction

 

The Streaming Performance Index and SPI Streams also appear as metrics in Trends and other dashboards.

Conviva formulates the unified streaming performance KPIs based on the percentage of streaming sessions with good or best viewing experience.

 

Each playback session can experience one or multiple QoE issues, for example a stream can have high rebuffering that leads to a playback failure.

When interpreting your data, keep in mind:

• Because an impacted stream can be associated with several improvement opportunities, such as high rebuffering leading to playback error (VPF-T), the percent of impacted sessions may not be the same as the sum of improvement opportunities percentages.

• SPI Streams show the Conviva monitored streaming sessions which contributed to the SPI calculation based on your SPI threshold settings. Sessions that exited before the video started when the viewer did not wait a significant time and sessions impacted by business logic errors are not included in the SPI streams.

• Abandonment and Startup Error are calculated as a percentage of all attempts, including attempts during which viewers had early abandonment.

• Plays show the video sessions during which the viewer saw at least the first frame of the video.

• Concurrent Plays show the maximum number of simultaneously active sessions during a given interval. Active sessions have at least one played video frame, and do not include unsuccessful play attempts or sessions that had ended.

SPI Analysis in Overview

In Overview, use the SPI to view a visual indicator of your KPIs and metric performance over the selected period. Also, quickly determine the number and percent of impacted streams. A stream is impacted when it fails to meet one or more of the KPI settings. The specific KPI performance of streams is shown in Improvement Opportunities.

In the Overview dashboard, click Streams, Viewers and Devices to toggle the display for analysis of impacts across video streams, viewers, and devices.

     

This data shows an hourly breakdown of the streams, viewers, and devices with at least 1 impacted session (SPI Stream) within the hour interval. Hover over an hourly interval to displayed the data details. Click an interval of Viewers to display the list of impacted viewers during that interval.

For streams, the engagement difference indicates a weighted comparison between the impacted stream playing times of good and bad sessions, with high engagement for a majority of good playing time and low engagement for a majority of bad playing time.

This is based on the following condition:

GoodSessionAverageLifePlayingTimeMins > BadSessionAverageLifePlayingTimeMins * 1.1

For example,

When there are impacts, the Impacted stream average playing time value is displayed. In this case, the GoodSessionAverageLifePlayingTimeMins is 22.8 and BadSessionAverageLifePlayingTimeMins is 5.72. 40.3% of the total streams where impacted.

The Impacted stream widget shows the value only when the number of good sessions exceeds bad sessions by 10 percent; else, it displays the message "Engagement difference low."

Impacted and non-impacted streaming times show the average playing times of the related sessions in minutes. For example, impacted streaming time is calculated as the sum of impacted session minutes in the current interval divided by the number of impacted sessions.

When significant engagement differences occur in stream playing time due to performance impacts, the streams tab also displays the average playing time of impacted and non-impacted streams.

Use a Conviva default set of criteria for comparison with other OTT streams based on satisfactory stream performance:

For metric definitions, see the Metric Dictionary.

Note: Clicking the SPI meter sorts the dimensional analysis by the SPI streams with color codes for the SPI index values based on the percentage for streams that pass the SPI KPI settings: above 80% green, between 80 and 60 percent yellow, below 60 percent red.

Improvement Opportunities

Improvement Opportunities Metrics:

High Rebuffering shows the percent of Ended Plays with a high Connection Induced Rebuffing Ratio (CIRR) or Connection Induced Rebuffering Time (CIRT).

High Startup Time shows the percent of Plays (at least 1 frame of video displayed) that took too long for the video to start.

Playback Error shows the percent of Ended Plays that ended due to a technical error (VPF-T).

Low Bitrate shows the percent of Ended Plays that had a low bitrate, as measured against different bitrate thresholds per screen size.

Startup Error shows the percent of all play attempts that failed to start due to a technical error (VSF-T).

Abandonment shows the percent of all play attempts that did not play after the user waited a long time for the video to start, by default set as EBVS wait time < 10sec for Conviva Good and EBVS wait time < 8sec for Conviva Best.

 

Note: Clicking an Improvement Opportunity sorts the dimensional analysis by that performance indicator, such as High Rebuffering.

Ads

Selecting Ads allows you to compare your overall startup and playback metrics across sessions with and without ads. Ads are broken out using sessions with and without pre-roll ads for the startup metrics, and sessions with and with mid-roll and post-roll ads for playback metrics.

Comparing sessions with and without ads requires Conviva Stream Sensor 4.x (Roku 3.x) with ad integrations.

Note: For server-side ads, a pre-roll ad can occur if a viewer joins a video and views an ad within 10 seconds of joining the video stream.

Use the Attempts and Ended Plays metrics to determine whether the level of traffic is sufficient.

Benchmark

Selecting the Benchmark icon displays your performance data and the 95th percentile performance across the selected cohort . The 95th percentile represents the value that exceeds 95% of the other values in the cohort distribution. The benchmark values represent the top performances across all publishers in the cohort and it's common that different publishers excel at different performance levels. In this example, the top 5% of the cohort had 1.48% or less sessions with high rebuffering.

For information on calculating the benchmarks, basic, and advance benchmarks, see Experience Overview.

Dimensional Analysis

Select dimensions to analyze your streaming performance across your most critical dimensions, such browsers, devices, and traffic categories. By default, dimensions are sorted by the number of streams and your SPI. You can also sort the dimension entries by clicking an Improvement Opportunities, such as High Rebuffering, as well as Plays and Concurrent Plays.

Click dimensional entries to create logical AND filters, such as Live in Content Category and Chrome in Browser Name to analyze Chrome Live traffic across the Cities dimension.

The Filter box displays the current filter settings.

Click a measure in the Improvement Opportunities widget to toggle the sort order by that measurement. In this example, the SPI performance highlights the Roku player and viewers in Arizona as under-performing due to high rebuffering during the selected period.

Based the Experience Overview highlights, use the Trends page for more detailed drill-downs or the Real-Time dashboard to set and monitor real-time session performance by key KPI thresholds.

SPI Analysis in Trends

In Trends, use the SPI score, summary metrics, and dimensional analysis. Clicking the SPI score or metric highlights that metric widget and displays the related metric data points in the dimension tables.

 

The dimension tables show the selected metric data, along with any previously selected metrics, in this case high rebuffering.

For more details about dimensional drill-downs and advanced filtering, see Trends.

 

SPI SPI Streaming Performance Index Streaming Performance Index Streaming Performance Index Abandonment SPI Streams wait time ebvs wait time ebvs wait time CIRT CIRT Connection Induced Rebuffering Time Connection Induced Rebuffering Time Low high bitrate Low High bitrate High High Rebuffering Playback Error Startup error VST-T startup time VPF-T