Build the Ushur Workflow

In this phase, the business process for classification and post-classification rules is designed and automated using Ushur’s no-code Flow Builder. Based on the training data prepared in the Data Collection phase, the workflow is created and tested by the business team to ensure the business KPIs are met. This can include using additional workflow-based optimizations to improve the solution accuracy beyond using just ML techniques. For example, use key business indicators such as member ID in the incoming email to improve the selected category’s accuracy further.

This phase involves the following steps:

• Workflow design using Ushur flow builder

• LI Training

• Evaluate Accuracy

Workflow Design

Prepare a simple workflow design before building the workflow in Ushur. During the workflow design, capture all the important components to be considered in the workflow. This visual document will ensure that the workflow is designed as per requirements. Here is an example of a simple use case workflow.

The Ushur team and you can start building the Ushur by logging into the dashboard and going to the Ushur builder. The Ushur platform has a very intuitive builder with simple drag and drop features. Check the following screenshot for reference:

LI Training

This is where the Ushur starts the learning process by consuming the training data. Based on the data prepared during the data collection phase, Ushur’s LI (language intelligence) starts the training. During the training process, the Ushur team will engage in a 90:10 split of the data (training: test data). Ushur uses high-precision tools to split the total data set into the training of the classification ML model and subsequent testing for analytics. Take a look at the following diagram:

There are multiple methods of initiating training, which can be referred from our SmartMail Integration Guide.

Evaluate Accuracy

What is to evaluate accuracy?

Testing the accuracy of Ushur classification and checking if the result is matching the Business requirement.

Why is it important?

You can test the power of the Ushur classification with an AI evaluator or UCV tool. AI evaluator can be used on the fly for testing single or lesser samples, and it’s available in the Ushur builder UI for easy access (select the “AI Mode”, which turns on the evaluator mode and you can click on AI on top of the Language Intelligence module to use the evaluator. Language Intelligence (LI) Module for more details). The other option of testing the accuracy is through Ushur’s UCV tool, with which more extensive sets of data can be tested. The UCV tool generates a confusion matrix report, which we can use to understand the model’s accuracy. You can run these steps to understand the accuracy.

Ushur team evaluates the accuracy based on the below parameters:

1. Precision = True Positive / (True Positive + False Positive)

2. Recall = True Positive / (True Positive + False Negative)

3. F1-Score = 2 * (Precision * Recall) / (Precision + Recall)

4. Overall accuracy = Correct prediction/total prediction

This is a critical stage, during which Ushur classification can be fine-tuned.

What tools and techniques are used?

UCV (Ushur Classification Verifier) and AI evaluator.

What do you need to do?

Run the UCV tool with test sample holdout set data which has the correct label mapping.

Who is the responsible party?

You and the Ushur team.

When should the process start?

As soon as the project starts.