Welcome toVigges Developer Community-Open, Learning,Share
Welcome To Ask or Share your Answers For Others


0 votes
in Technique[技术] by (71.8m points)

precision recall - How do I define a custom eval metric in Catboost (python) compatible with class_weights?

Using this example, I created a precision-recall AUC eval metric for Catboost. However, I need some guidance on how to make it compatible with the class_weights argument in which I will be passing a list (example: [625.0, 0.500400320256205]) because I have a large class imbalance. Below is my custom eval metric:

from sklearn.metrics import average_precision_score
import numpy as np
from scipy.special import expit

# define pr-AUC custom eval metric
class PrecisionRecallAUC:
    # define a static method to use in evaluate method
    def get_pr_auc(y_true, y_pred):
        # fit predictions to logistic sigmoid function
        y_pred = expit(y_pred).astype(float)
        # actual values should be 1 or 0 integers
        y_true = y_true.astype(int)
        # calculate average precision
        flt_pr_auc = average_precision_score(y_true=y_true, 
        return flt_pr_auc
    # define a function to tell catboost that greater is better (or not)
    def is_max_optimal(self):
        # greater is better
        return True

    # get the score
    def evaluate(self, approxes, target, weight):
        # make sure length of approxes == 1
        assert len(approxes) == 1
        # make sure length of target is the same as predictions
        assert len(target) == len(approxes[0])
        # set target to integer and save as y_true
        y_true = np.array(target).astype(int)
        # save predictions
        y_pred = approxes[0]
        # generate score
        score = self.get_pr_auc(y_true=y_true, y_pred=y_pred)
        return score, 1

    # return score
    def get_final_error(self, error, weight):
        return error

How do I use my list of class weights in this custom eval metric?

Thank you in advance.

Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Welcome to Vigges Developer Community for programmer and developer-Open, Learning and Share