def estimate_cont_entropy(X, epsilon=None):
    """Estimate the Shannon entropy of a discrete dataset.

    Based off the Kraskov Estimator [1] and Kozachenko [2]
    estimators for a dataset's Shannon entropy.

    The function relies on nonparametric methods based on entropy
    estimation from k-nearest neighbors distances as proposed
    in [1] and augmented in [2] for mutual information estimation.

    If X's columns logically represent discrete features,
    it is better to use the calculate_disc_entropy function.
    If you are unsure of which to use, estimate_entropy can
    take datasets of mixed discrete and continuous functions.

    Args:
        X (array-like): An array-like (np arr, pandas df, etc.) with shape
            (n_samples, n_features) or (n_samples)
        epsilon (array-like): An array with shape (n_samples, 1) that is
            the epsilon used in Kraskov Estimator. Represents the chebyshev
            distance from an element to its k-th nearest neighbor in the full
            dataset.

    Returns:
        float: A floating-point number. If epsilon is not provided,
            this will be the Kozacheko Estimator of the dataset's entropy.
            If epsilon is provided, this is a partial estimation of the Kraskov
            entropy estimator. The bias is cancelled out when computing
            mutual information.

    References:

    .. [1] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual
           information". Phys. Rev. E 69, 2004.
    .. [2] L. F. Kozachenko, N. N. Leonenko, "Sample Estimate of the Entropy
           of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16

    """
    X = asarray2d(X)
    n_samples, n_features = X.shape
    if n_samples <= 1:
        return 0
    nn = NearestNeighbors(
        metric='chebyshev',
        n_neighbors=NUM_NEIGHBORS,
        algorithm='kd_tree')
    nn.fit(X)
    if epsilon is None:
        # If epsilon is not provided, revert to the Kozachenko Estimator
        n_neighbors = NUM_NEIGHBORS
        radius = 0
        # While we have non-zero radii, calculate for a larger k
        # Potentially expensive
        while not np.all(radius) and n_neighbors < n_samples:
            distances, _ = nn.kneighbors(
                n_neighbors=n_neighbors, return_distance=True)
            radius = distances[:, -1]
            n_neighbors += 1
        if n_neighbors == n_samples:
            # This case only happens if all samples are the same
            # e.g. this isn't a continuous sample...
            raise ValueError('Should not have discrete column to estimate')
        return -digamma(n_neighbors) + digamma(n_samples) + \
            n_features * np.mean(np.log(2 * radius))
    else:
        ind = nn.radius_neighbors(
            radius=epsilon.ravel(),
            return_distance=False)
        nx = np.array([i.size for i in ind])
        return - np.mean(digamma(nx + 1)) + digamma(n_samples)
def estimate_cont_entropy ( X , epsilon = None ) : X = asarray2d ( X ) n_samples , n_features = X . shape if n_samples <= 1 : return 0 nn = NearestNeighbors ( metric = 'chebyshev' , n_neighbors = NUM_NEIGHBORS , algorithm = 'kd_tree' ) nn . fit ( X ) if epsilon is None : n_neighbors = NUM_NEIGHBORS radius = 0 while not np . all ( radius ) and n_neighbors < n_samples : distances , _ = nn . kneighbors ( n_neighbors = n_neighbors , return_distance = True ) radius = distances [ : , - 1 ] n_neighbors += 1 if n_neighbors == n_samples : raise ValueError ( 'Should not have discrete column to estimate' ) return - digamma ( n_neighbors ) + digamma ( n_samples ) + n_features * np . mean ( np . log ( 2 * radius ) ) else : ind = nn . radius_neighbors ( radius = epsilon . ravel ( ) , return_distance = False ) nx = np . array ( [ i . size for i in ind ] ) return - np . mean ( digamma ( nx + 1 ) ) + digamma ( n_samples )
Estimate the Shannon entropy of a discrete dataset.

    Based off the Kraskov Estimator [1] and Kozachenko [2]
    estimators for a dataset's Shannon entropy.

    The function relies on nonparametric methods based on entropy
    estimation from k-nearest neighbors distances as proposed
    in [1] and augmented in [2] for mutual information estimation.

    If X's columns logically represent discrete features,
    it is better to use the calculate_disc_entropy function.
    If you are unsure of which to use, estimate_entropy can
    take datasets of mixed discrete and continuous functions.

    Args:
        X (array-like): An array-like (np arr, pandas df, etc.) with shape
            (n_samples, n_features) or (n_samples)
        epsilon (array-like): An array with shape (n_samples, 1) that is
            the epsilon used in Kraskov Estimator. Represents the chebyshev
            distance from an element to its k-th nearest neighbor in the full
            dataset.

    Returns:
        float: A floating-point number. If epsilon is not provided,
            this will be the Kozacheko Estimator of the dataset's entropy.
            If epsilon is provided, this is a partial estimation of the Kraskov
            entropy estimator. The bias is cancelled out when computing
            mutual information.

    References:

    .. [1] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual
           information". Phys. Rev. E 69, 2004.
    .. [2] L. F. Kozachenko, N. N. Leonenko, "Sample Estimate of the Entropy
           of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16