metrics¶
Different Metric functions.
-
deeppavlov.metrics.accuracy.
sets_accuracy
(y_true: [<class 'list'>, <class 'numpy.ndarray'>], y_predicted: [<class 'list'>, <class 'numpy.ndarray'>]) → float[source]¶ Calculate accuracy in terms of sets coincidence
Parameters: - y_true – true values
- y_predicted – predicted values
Returns: portion of samples with absolutely coincidental sets of predicted values
-
deeppavlov.metrics.accuracy.
classification_accuracy
(y_true: List[list], y_predicted: List[Tuple[list, dict]]) → float[source]¶ Calculate accuracy in terms of sets coincidence for special case of predictions (from classification KerasIntentModel)
Parameters: - y_true – true labels
- y_predicted – predictions. Each prediction is a tuple of two elements (predicted_labels, dictionary like {“label_i”: probability_i} )
Returns: portion of samples with absolutely coincidental sets of predicted values
-
deeppavlov.metrics.fmeasure_classification.
classification_fmeasure
(y_true: List[list], y_predicted: List[Tuple[list, dict]], average='macro') → float[source]¶ Calculate F1-measure macro
Parameters: - y_true – true binary labels
- y_predicted – predictions. Each prediction is a tuple of two elements (predicted_labels, dictionary like {“label_i”: probability_i} ) where probability is float or keras.tensor
- average – determines the type of averaging performed on the data
Returns: F1-measure
-
deeppavlov.metrics.fmeasure_classification.
classification_fmeasure_weighted
(y_true: List[list], y_predicted: List[Tuple[list, dict]], average='weighted') → float[source]¶ Calculate F1-measure weighted
Parameters: - y_true – true binary labels
- y_predicted – predictions. Each prediction is a tuple of two elements (predicted_labels, dictionary like {“label_i”: probability_i} ) where probability is float or keras.tensor
- average – determines the type of averaging performed on the data
Returns: F1-measure
-
deeppavlov.metrics.log_loss.
classification_log_loss
(y_true: List[list], y_predicted: List[Tuple[list, dict]]) → float[source]¶ Calculate log loss for classification module
Parameters: - y_true – true binary labels
- y_predicted – predictions. Each prediction is a tuple of two elements (predicted_labels, dictionary like {“label_i”: probability_i} ) where probability is float or keras.tensor
Returns: log loss
-
deeppavlov.metrics.roc_auc_score.
classification_roc_auc_score
(y_true: List[list], y_predicted: List[Tuple[list, dict]]) → float[source]¶ Compute Area Under the Curve (AUC) from prediction scores.
Parameters: - y_true – true binary labels
- y_predicted – predictions. Each prediction is a tuple of two elements (predicted_labels, dictionary like {“label_i”: probability_i} )
Returns: Area Under the Curve (AUC) from prediction scores