Compute precision and recall at a given cutoff for each user. In information
    retrieval terms, precision represents the ratio of relevant, retrieved items
    to the number of relevant items. Recall represents the ratio of relevant,
    retrieved items to the number of relevant items.

    Let :math:`p_k` be a vector of the first :math:`k` elements in the
    recommendations for a particular user, and let :math:`a` be the set of items
    in ``observed_user_items`` for that user. The "precision at cutoff k" for
    this user is defined as

    .. math::
        P(k) = \\frac{ | a \cap p_k | }{k},

    while "recall at cutoff k" is defined as

    .. math::
        R(k) = \\frac{ | a \cap p_k | }{|a|}

    The order of the elements in the recommendations affects the returned
    precision and recall scores.

    Parameters
    ----------
    observed_user_items : SFrame
        An SFrame containing observed user item pairs, where the first
        column contains user ids and the second column contains item ids.

    recommendations : SFrame
        An SFrame containing columns pertaining to the user id, the item id,
        the score given to that pair, and the rank of that item among the
        recommendations made for user id. For example, see the output of
        recommend() produced by any turicreate.recommender model.

    cutoffs : list[int], optional
        The cutoffs to use when computing precision and recall.

    Returns
    -------
    out : SFrame
        An SFrame containing columns user id, cutoff, precision, recall, and
        count where  the precision and recall are reported for each user at
        each requested cutoff, and count is the number of observations for
        that user id.

    Notes
    -----
    The corner cases that involve empty lists were chosen to be consistent
    with the feasible set of precision-recall curves, which start at
    (precision, recall) = (1,0) and end at (0,1). However, we do not believe
    there is a well-known consensus on this choice.

    Examples
    --------
    Given SFrames ``train_data`` and ``test_data`` with columns user_id
    and item_id:

    >>> from turicreate.toolkits.recommender.util import precision_recall_by_user
    >>> m = turicreate.recommender.create(train_data)
    >>> recs = m.recommend()
    >>> precision_recall_by_user(test_data, recs, cutoffs=[5, 10])

def precision_recall_by_user(observed_user_items,
                             recommendations,
                             cutoffs=[10]):
    """
    Compute precision and recall at a given cutoff for each user. In information
    retrieval terms, precision represents the ratio of relevant, retrieved items
    to the number of relevant items. Recall represents the ratio of relevant,
    retrieved items to the number of relevant items.

    Let :math:`p_k` be a vector of the first :math:`k` elements in the
    recommendations for a particular user, and let :math:`a` be the set of items
    in ``observed_user_items`` for that user. The "precision at cutoff k" for
    this user is defined as

    .. math::
        P(k) = \\frac{ | a \cap p_k | }{k},

    while "recall at cutoff k" is defined as

    .. math::
        R(k) = \\frac{ | a \cap p_k | }{|a|}

    The order of the elements in the recommendations affects the returned
    precision and recall scores.

    Parameters
    ----------
    observed_user_items : SFrame
        An SFrame containing observed user item pairs, where the first
        column contains user ids and the second column contains item ids.

    recommendations : SFrame
        An SFrame containing columns pertaining to the user id, the item id,
        the score given to that pair, and the rank of that item among the
        recommendations made for user id. For example, see the output of
        recommend() produced by any turicreate.recommender model.

    cutoffs : list[int], optional
        The cutoffs to use when computing precision and recall.

    Returns
    -------
    out : SFrame
        An SFrame containing columns user id, cutoff, precision, recall, and
        count where  the precision and recall are reported for each user at
        each requested cutoff, and count is the number of observations for
        that user id.

    Notes
    -----
    The corner cases that involve empty lists were chosen to be consistent
    with the feasible set of precision-recall curves, which start at
    (precision, recall) = (1,0) and end at (0,1). However, we do not believe
    there is a well-known consensus on this choice.

    Examples
    --------
    Given SFrames ``train_data`` and ``test_data`` with columns user_id
    and item_id:

    >>> from turicreate.toolkits.recommender.util import precision_recall_by_user
    >>> m = turicreate.recommender.create(train_data)
    >>> recs = m.recommend()
    >>> precision_recall_by_user(test_data, recs, cutoffs=[5, 10])
    """

    assert type(observed_user_items) == _SFrame
    assert type(recommendations) == _SFrame
    assert type(cutoffs) == list
    assert min(cutoffs) > 0, "All cutoffs must be positive integers."
    assert recommendations.num_columns() >= 2
    user_id = recommendations.column_names()[0]
    item_id = recommendations.column_names()[1]

    assert observed_user_items.num_rows() > 0, \
           "Evaluating precision and recall requires a non-empty " + \
           "observed_user_items."
    assert user_id in observed_user_items.column_names(), \
            "User column required in observed_user_items."
    assert item_id in observed_user_items.column_names(), \
            "Item column required in observed_user_items."
    assert observed_user_items[user_id].dtype == \
           recommendations[user_id].dtype, \
           "The user column in the two provided SFrames must have the same type."
    assert observed_user_items[item_id].dtype == \
           recommendations[item_id].dtype, \
           "The user column in the two provided SFrames must have the same type."

    cutoffs = _array.array('f', cutoffs)

    opts = {'data': observed_user_items,
            'recommendations': recommendations,
            'cutoffs': cutoffs}

    response = _turicreate.toolkits._main.run('evaluation_precision_recall_by_user', opts)
    sf = _SFrame(None, _proxy=response['pr'])
    return sf.sort([user_id, 'cutoff'])