Verifying Integration Reports

The Reports dashboard displays validation test results of metrics and metadata for all of your device applications.
You can choose to verify the validation results for either metrics or metadata for all or selected apps.

Using the Reports dashboard, view reports for:

  • App Statistics

  • Integration Overview

 

Use App Statistics report to learn about:

  • The performance of metrics and metadata for all or specific apps

  • Apps that need the most attention (appearing first on the list)

  • Last validation date of the apps

  • Newest build or SDK version if available

App Statistics Report

  1. Navigate to the Reports tab.

  2. Select App Statistics as report type from the Show dropdown menu.

  3. Select either All Apps or a specific app from the dropdown menu.

    You can choose to hide certain apps from the report using the Hide Apps option.


  4. Verify the validation results for metrics, metadata, and SDK versions.

    The Metrics, Metadata, and SDK & Validation tabs list the apps such that the apps requiring the most attention are displayed first.

    To improve the Streaming Performance Index (SPI) and data accuracy in Pulse, fix the under-performing metrics and metadata highlighted in the Need Attention field. To learn additional details about the failed metrics or metadata and propsed fixes, refer to the Notes column in the validation report. For more information about self-validating the builds after fixing the issues, downloading the test cases, and watching sample validation videos, refer to the Self-Validation page on the Learning Center.

    Metrics Report

    Display order:
    Content Metrics - Must Have --> Content Metrics - Should Have --> Ad Metrics - Must Have and --> Ad Metrics - Should Have
    This report focuses on computing the Must-have content metrics to highlight the under performing ones in the Need Attention field. Fix these impacted metrics before moving on to the Metadata tab.

    Metadata Report

    The metadata report displays the Required and Custom metadata with the status Pass, Fail, or N/A. Identify the custom metadata type by the tag, Content or Ad, adjacent to the metadata key.

    SDK & Validation Details

    This page lists the apps with their last Validated On date, and the newest SDK version if available. The hyperlink on the version number redirects you to the corresponding platform integration page on the Learning Center. To download and integrate the latest SDK version, refer to the instructions on the Learning Center page.

  5. To download the reports (in Excel format) , select the Status Report Download option and choose the apps.

Integration overview reports help to analyze the integration details and the validation improvement progress for each app for a selected time period of 3 months, 6 months, 9 months, or 12 months. These reports use graphical representations to show the impact or improvement of metrics and metadata.

Integration Overview Report

  1. Navigate to the Reports tab.

  2. Select Integration Overview from the Show dropdown menu.

  3. Select the desired app from the App Name dashboard menu.

    Note: You can generate the Integration Overview report for one app at a time; not for all available apps.

  4. Choose the time period.

  5. This report provides the details such as:

Section Description Examples
App Submission Distribution This graph helps to understand the ratio of builds submitted to production versus builds validated with Conviva.

As per this graph, only one build was sent for validation against 70 builds that were deployed into production in the last 6 months; hence, there are high chances of issues with their data available on Pulse.
Metrics and Metadata Test Results This graph helps to compare the builds for a particular time period.

Metrics Test Results: This graph shows that the last build validated was in December, not sent to production, and passed only five test cases.

 

 
Metrics and Metadata - Impact Heatmap

  • The heatmap helps to analyze the metrics and metadata that have been failing over time and need more attention to reduce the failures.

  • The data division is based on percentages.

  • Shows both positive and negative impact.

Metrics Impact Heatmap
In this example, there's a recurring issue with EBVS and Attempts. Until they're fixed, they impact the data available on Pulse.

Metadata Impact Heatmap
Here, the Device Category has the major impact, and it means that the device metadata doesn't work properly for this app.

 

This feature enables mapping the Validated - Not In Production builds against the Not validated - In production builds.

Video Integration tracks the performance of metrics and metadata as part of the build validation process. If builds are directly deployed in production without being validated, it's difficult to know the exact impacted metrics and failed metadata. There can be instances where the validated build and the direct production-deployed build are technically similar with different build versions. In such cases, mapping these builds enables the data availability in Video Integration.

Build Mapping

  1. Navigate to the Reports tab, select the Integration Overview report, and choose an application to generate the report.

    For example, the selected app Roku | Exec Report has a Validated - Not in production build and a Not validated - In production build as shown in the App Submission Distribution chart.


    The Test Results - Metrics graph shows the details for these two builds separately, where in the not-validated build doesn't have any details in Video Integration.

  2. Select the Build Mapping option.

  3. Select the corresponding validated and not-validated builds.

  4. Click Apply to map the builds. The graphs now show only the mapped In Production build. You can choose to detach the mapped builds.

    App Submission Distribution

    Test Results - Metrics