Evaluate Model

Evaluate Model

Model evaluation in Skyl.ai allows you to evaluate your text multiclass models using a test dataset.  You can automatically compare how well your models are performing by to the true data. This can be done through uploading a completed .csv and viewing the test results by clicking on the dataset.

  1. Select ‘Evaluate Model’ under Machine Learning

  2. Select ‘Start a New Model Evaluation’ on the top right-hand corner of the screen’

  3. Select which deployed model you would like to test and evaluate under “Select a Deployed Model”

  4. Check that your.csv file has the proper schema by selecting “Download Model Evaluation Schema” above the blue box

  5. Upload your .csv file by dragging and dropping into the blue box or choosing it from a local file

  6. Preview your .csv file before evaluation process

  7. Click “Start Evaluation” to start the evaluation process or “Cancel” to cancel the evaluation

  8. Wait until evaluation process will be completed

  9. Review prediction confidence, evaluation result, performance metrics, and class metrics

  10. Download evaluation result as .csv file

Model Inference

Model Inference

Once the training has been completed, a machine learning model would be created which will list under Models on ‘Model Deployment’ page. You can view training summary reports of the model.
In order to start leveraging the power of your newly created model, you need to deploy it first.

To deploy the model go to the ‘Deployment’ tab and select the ‘Deploy’ button

After deploying the Model, Inference API will be available which can be integrated into your application.

Model Training

Model Training

Skyl provides relevant and optimized algorithms to train ML models depending on the ML template. A Text Classification Multiclass model on Skyl is a Natural Language Processing model that classifies or summarizes the idea behind text.

How to train an ML model for Text Classification Multiclass:

  1. Select “Training” under Machine Learning. You will be redirected to the ‘Train’ page

  2. Select ‘Train a new Model’ to start configuring a new model to train

  3. Name the new model and add description.

  4. Select a featureset

  5. Review the featureset details and check your selection conditions.

  6. Select and configure the algorithm.

  7. Select the ‘Train’ button to begin the training process

Initiated training will be displayed on the ‘Training’ page

As the training is in process, user can see the training logs in real-time under the Logs tab.

Creating a Featureset

Creating a Featureset

A featureset is a subset of the dataset with text and their classes which is ultimately used as input for training models.  Using the entire dataset for models can lead to inaccuracy and a higher chance of using unclean data, so we created the featureset to ensure accuracy in more models.

How to create a featureset in text classification multiclass:

  1. Select Machine Learning from the left menu and select “Featureset”. From there, you will be directed to the ‘Featureset’ page

  2. On the top right corner, select “Add Featureset”

  3. Name your new featureset and add its description

  4. From the drop-down menu, select which dataset the featureset will be selected from

  5. On the ‘Feature Selection’ tab, select the categories from the ‘collect’ column. Then, label the column or merge* the collect and label columns

    * Not empty records from label column replace records in collect column.

  6. Define and train the test sets. Skyl provides two options for defining and training test sets. A user can either split the dataset, or explicitly extract from the dataset

    Define the train and test sets with the ‘Split the dataset’ policy:

    1. Choose the selection method from which rows are selected

    2. Enter the number* of records for selection.

      *the number of selected images cannot be bigger than the number of records in dataset

    3. Enter the train ratio (from 0.01 to 1). The train ratio is the proportion of selection rows versus columns that go into the training set. The rest of the images go to the test set

    4. Enter a random seed so that you can reproduce results

    Define train and test sets with the ‘Explicit Extract from the Dataset’ policy:

    1. Choose the selection method and number of records under ‘Train Set’

    2. Enter the number of records and choose selection method test set

    3. Optional- Apply a filter to train and test sets so that you can avoid any unwanted records in the selection process.

    4. Enter a random seed so that you can reproduce results

  7. Select ‘Create Featureset’ to create a new featureset

All created featuresets are displayed on the ‘Featureset’ page.

Each featureset has a status of creation:

  • ‘Completed’ means the featureset is ready to be used for ML training
  • ‘In Progress’ means tells that featureset is being prepared
  • ‘Failed’ means the featureset wasn’t created

Open the Featureset details slider by selecting the proper featureset to see the details and status of each Featureset