Metric Builder

Metric Builder enables you to build flows and custom metrics associated with specified initial/follow-up events to analyze data for complex user actions and app performance scenarios, such as analyzing the amount of time spent between viewer login and the start of video playtime and the success rate of cart uploads to completed purchase.

  • Flows: helps in the identification of crucial moments in user journeys to optimize applications and improve the user experience. For example, build the flow from searching for a video to successfully playing it, and use the corresponding metrics to analyze the flow.

  • Custom metrics associated with initial and follow up events: ECO enables three types of custom metrics.

    • Duration Metric: Measures the time interval between two events, such as the duration between the login screen event and the login success event.

    • Conversion Metric: Measures the count or conversion rates of events from one stage to another, such as the count of sessions with successful login after opening the login screen.

    • Event Aggregation Metric: Measures the number of events, or calculates the value of event attributes, such as the average duration of network requests.

Metric Builder provides templates for guided metric creation, cloning existing metrics, and building metrics based on specified settings.

After creating custom metrics and selecting the metrics from the Select metrics to view page, the created metrics appear as ECO widgets with time series and dimension data to enable deeper analysis using metric filtering and dimensional drill-down. For example, the custom metric Login to Purchase shows the time duration between login and successful purchase. This metric enables deeper analysis regarding the customer purchase behavior and allows you to optimize the sales process, enhance user experience, and boost customer loyalty.

Note: To enhance the analysis of custom metrics, ECO displays secondary custom metrics in the dimension tables using <metric_name> Init# for conversion metrics and <metric_name> Complete# for duration metrics. <metric_name> Init# shows the number of times the initial event was performed; <metric_name> Complete# shows the number of times both the initial and follow up events were performed, based on the successful and failed pairs from the first initial event to the first follow-up event.

Creating Flows

Conviva ECO enables flows that reflect user journeys, helping in the identification of crucial moments to optimize applications and improve the user experience. For example, the 'search for playing videos' flow highlights the user experience from searching for a video to successfully playing it. When setting up a flow, ECO automatically generates success, error, and duration metrics. Use these metrics within a flow widget to monitor core business scenarios at a glance.

Note: When proceeding with the steps, check the Guide on the right side for reference.

  1. Click + New Metric and select Create Flow to create flow-based metrics.

  2. Enter the flow name in the Flow Name field.

    For example, to track the flow from searching to playing videos, enter 'search for playing videos'.

  3. Select events for flow creation:

    1. If you are unsure about the exact events to select, use the tools to identify them:

      • LiveLens: Provides real-time event names and tags while interacting with an application. To learn how to use LiveLens, see LiveLens.

      • EventLooker: Allows browsing and searching events and tag keys.

    2. Select or map new events for flow creation:

      1. Starting Event: Marks the start of a flow.

        For example, to create metrics that measure a user journey that starts with search, select the 'Search Submit' event, which occurs when the user clicks the submit button.

      2. Success Event: Marks the end of a flow.

        For example, to end a user journey flow when the viewer watches a video, select the ‘Video Play’ event, which occurs when the user clicks the play button.

      3. Error Event (Optional): Tracks errors in a flow.

        If not selected, the application does not generate the error metric.

        For example, to track errors in the user journey related to conviva defined errors, select the ‘conviva_defined_error’ event, which occurs when errors happen in the flow.

        Note:  To enable additional events for flow-based metrics, map raw events. For more details, see Map New.

  4. Define flow details, such as time constraint, distribution range, and rating limits.

    1. Set the timeframe during which flow success and errors are determined.

      For example, to evaluate flow performance, set a timeframe of 120 seconds during which metrics are assessed.

    2. Enter percentage values to qualify the values as poor or good performance.

      For example, to assess performance quality, define values less than or equal to 20% as poor performance, while values greater than or equal to 80% indicate good performance.

    3. Define the expected value range for durations to display on the distribution chart. ECO fills out the range automatically after setting the timeframe.

      For example, to visualize acceptable user interaction times, set the duration range from 20 seconds to 200 seconds.

  5. Click Save&Validate and ECO creates the corresponding metrics automatically. Check the generated metrics in a preview version with sample data, and make updates to the settings if necessary.

    • Metric: <flow_name> Init

    • Metric: <flow_name> Complete

    • Metric: <flow_name> Avg Duration

    • Metric: <flow_name> Success #

    • Metric: <flow_name> Success %

    • Metric: <flow_name> Error

    Note: Hovering over the Question Mark icon displays more details about the generated flow metrics.

       
    Metric: <flow_name> Init Counts the number of initial events. In this example, users 1, 2, 3, and 4 perform the initial event, so the metric value is 4.
    Metric: <flow_name> Complete

    Counts the number of initial events that ended with a complete event within the flow, not subject to the flow time constraint.

    In this example, users 1, 2, and 4 performed both the initial and complete events, so the metric value is 3.
    Metric: <flow_name> Avg Duration Shows the sum of complete event durations divided by the number of complete flows. The metric is not subject to constraints. Based on the duration for flow events from users 1, 2, and 4, the metric value is (t1+t2+t4) / 3
    Metric: <flow_name> Success #

    Counts the number of initial events that ended with complete events within the flow, including the flow time constraint.

    In this example, users 1 and 4 performed the initial and complete events within the flow constraints, so the metric value is 2.
    Metric: <flow_name> Success %

    Shows the percentage of successful flows, determined by number of initial events with and complete events within the flow constraints divided by the total number of initial events.

    In this example, users 1 and 4 completed successful flow, while all 4 users performed the initial event, so the metric value is 50%.
    Metric: <flow_name> Error

    Counts the number of flows with errors that occur within the flow time constraint.

    In this example, only users 1 and user 4 encountered errors during flows time constraints, so the metric value is 2.

  6. Click Deploy to use the metrics for analysis on the Trends dashboard.

    For example, after deploying the metrics based on the 'search for playing videos' flow, go to the Trends dashboard to review the metrics and identify key insights for optimizing application performance and improving the user experience. An increase in the duration between the initial and complete events indicates slower performance while a decrease in success % indicates issues or blockers to user journey completion within the time constraint.

Creating Custom Metrics Associated With Initial and Follow Up Events

Using Templates to Create Metrics

Use case templates enable efficient metric creation with the assistance of pre-filled fields and commonly used event associations. The use case templates provide clear guidance on where to begin and establish a well-defined process for metric creation. Common use cases involve measuring the time required to complete a user login, a successful checkout, and other time-critical user experiences.

  1. To browse all use cases, select Conviva Metric Templates from the Metric Builder page.

  2. From the use case list, select a required use case. In this case, choose Login Start to Complete Avg Time. You can check comprehensive information to better understand the specifics of each metric after selecting.

  3. Select either of the options to create a metric:

    • Create This Metric with Wizard: Choose this option to automatically pre-fill most of the configuration information.

      Note: You cannot update the prefilled information.

    • Clone It for Manual Metric Creation: Choose this option for more flexibility in customization, with the prefilled information as a reference. This option allows you to tailor the metric to your specific needs.

      Create This Metric with Wizard:

         

      1. Find an appropriate event from the event lists, add the event to the Sematic Mapper and click Create, Next Step to proceed. In this example, use conviva_network_request to represent the Login Process Start.

      2. Follow the same process. In this example, use screen_view to represent the Login Response Complete.

      3. Preview all the fields, update the words in the Description field if necessary, and click Create. You can find the created metric in the User Defined list.

      Clone It for Manual Metric Creation

         

      1. Edit all the prefilled messages based on your specific requirements.

      2. To save your updates, click Save. You can find the created metric in the User Defined list.

       

Creating Fully Customized Metrics

To create a custom metric using Metric Builder:

  1. Click + New Metric and select Create Individual Metrc.

  2. Enter the basic metric information.

    1. Metric Name

    2. Metric Description

  3. From the Metric Type list, select the desired metric type.

    • Duration Metric: Measures the time interval between two events.

    • Conversion Metric: Measures the count or conversion rates of events from one stage to another.

      Displays the count of sessions with successful conversions or the count of successful attempts.

    • Event Aggregate Metric: Measures the number of events, or calculates the value of event attributes, such as the average duration of network requests.

  4. In the event selection and detail fields, select the events and values based on your Metric Type selection:

    1. If you are unsure about the exact events to select, use the tools to identify them:

      • LiveLens: Provides real-time event names and tags while interacting with an application. To learn how to use LiveLens, see LiveLens.

      • EventLooker: Allows browsing and searching events and tag keys.

    2. Select or map new events for metric creation:

      Note:  To enable additional events for custom metrics, map raw events. For more details, see Map New.

      • For Duration metrics:

        Enter the initial and follow-up events along with the pairing logic.

        • Initial Event: The first event that triggers a particular process or workflow. It serves as the starting point for measuring and tracking user interactions or system behavior.

        • Follow-up Event: A subsequent event that occurs after the Initial Event. It is a continuation or a result of the initial action. The Follow-up Event helps track and measure user engagement or system behavior beyond the initial interaction.

          For example, to measure the average duration from login start to successful login, select Login Process Start and Login Success.

        • Pairing Logic:

               Pairing Logic

          As the same events can appear multiple times in an app process, pairing logic specifies the first or last sequential instance of the initial and follow-up mapped events across multiple event instances.

          • First-First Pair: Uses the first Initial Event and the first Follow-up Event as a pair.

          • Last-First Pair: Uses the last Initial Event and the first Follow-up Event as a pair.

          • Count in the same session: Choose either first pair or multiple pairs.

            • first pair selects only the first pair in a session.

            • multiple pairs selects all pairs that meet the pairing logic in the same session.

              Pairing Logic

              For example, suppose you have two mapped events: Login Screen View indicates viewers opening the login page, and Login Success indicates viewers successfully logging into their accounts. In a session, the events are displayed in the following time order: Login Screen View1, Login Screen View2, Login Screen View3, Login Success1, Login Screen View4, Login Success2.

                first-first pair last-first pair If selecting first-first pair and first pair If selecting first-first pair and multiple pairs If selecting last-first pair and first pair If selecting last-first pair and multiple pairs
              Event name Login Screen View1 and Login Success1 Login Screen View3 and Login Success1 Login Screen View1 and Login Success1 Login Screen View1 and Login Success1; Login Screen View4, Login Success2 Login Screen View3 and Login Success1 Login Screen View3 and Login Success1; Login Screen View4, Login Success2

            • (Optional) To show the distribution of the metric in ECO, select the Display the distribution of this metric check box.

               

      • For Conversion metrics:

        Enter the initial and follow-up events along with the conversion rate definition.

        • Initial Event: This refers to the initial event as defined previously.

        • Follow-up Event: This refers to the follow-up event as defined previously.

          For example, to measure the conversion rate from login start to successful login, select Login Process Start and Login Success.

        • Define the details of the metric.

          • Select the conversion type, Conversion Rate or Conversion Count

          • Calculate Conversion Rate / Conversion Count of successful conversions within a given seconds/minutes/hours: Set a criteria for the duration between the two events. For example, if you set Calculate Conversion Count of successful conversions within 30 seconds and the Initial Event is Login Screen View and the Follow-up Event is Login Success, the criteria is that the two events must occur within 30 seconds to be considered a successful conversion.

          • Select an option from the drop-down list to define whether a higher or lower metric value is considered positive. For example, choosing Higher is better indicates that a higher value is desirable, whereas choosing Lower is better indicates that a lower value is desirable.

      • For Event Aggregation metrics:

        • Select the source aggregation event and define the type of aggregation.

          For example, to measure the total number of application errors, select Application Error.

        • Define the details of the metric.

          • Set the value calculation type

            • Number: Sets the value to the count of the source events.

            • Total Values: Sets the value to the sum of the event attribute values.

            • Average Values: Sets the value to the averge or mean of the event attribute values.

          • Select an option from the drop-down list to define whether a higher or lower metric value is considered positive. For example, choosing Higher is better indicates that a higher value is desirable, whereas choosing Lower is better indicates that a lower value is desirable.

          • (Optional) To show the unit of the metric in the ECO Trends dashboard, select the Display unit in Trends check box.

  5. Click Save&Validate and the customized metric you created displays in a preview version with sample data, and make updates to the settings if necessary.

  6. Click Deploy to complete the deployment.

    The deployment takes several minutes to take effect.

Preview

The Preview function allows you to check and validate undeployed metrics and dimensions with up to 14 days of historical sampled data, ensuring metric composition and effectiveness before deployment.

  1. From the Metric Builder tab, click Preview in the top right corner.

  2. On the opened Preview page, click the Query range icon, select the query time range and click Apply.

  3. Check Metrics and Dimensions from metric widgets and dimension tables.

    Note: A Undeployed label appears next to the undeployed metrics and dimensions.

  4. Based on the check results, update or deploy the metrics and dimensions as required. To deploy the metrics or dimensions, click the Deploy button in the top right corner, select the metrics and dimensions based on requirements, and click Deploy.

ECO Custom Metrics ECO Custom Metrics ECO Metric Template ECO Metric Template metric builder metric builder Preview Preview flow flow flow metrics flow metrics