Print the statistics.

    :param completion: the completion of each duplicated markers.
    :param concordance: the pairwise concordance between duplicated markers.
    :param tped: a representation of the ``tped`` of duplicated markers.
    :param snps: the positions of the duplicated markers in the ``tped``
    :param frequencies: the frequency of each of the duplicated markers.
    :param prefix: the prefix of the output files.
    :param diffFreq: the frequency difference threshold.

    :type completion: numpy.array
    :type concordance: dict
    :type tped: numpy.array
    :type snps: dict
    :type frequencies: dict
    :type prefix: str
    :type diffFreq: float

    :returns: a :py:class:`set` containing duplicated markers to complete.

    Creates a summary file (``prefix.summary``) containing information about
    duplicated markers: chromosome, position, name, alleles, status, completion
    percentage, completion number and mean concordance.

    The frequency and the minor allele are used to be certain that two
    duplicated markers are exactly the same marker (and not a tri-allelic one,
    for example).

    For each duplicated markers:

    1. Constructs the set of available alleles for the first marker.
    2. Constructs the set of available alleles for the second marker.
    3. If the two sets are different, but the number of alleles is the same, we
       try to flip one of the marker. If the two sets are the same, but the
       number of alleles is 1, we set the status to ``homo_flip``. If the
       markers are heterozygous, we set the status to ``flip``.
    4. If there is a difference in the number of alleles (one is homozygous,
       the other, heterozygous), and that there is on allele in common, we set
       the status to ``homo_hetero``. If there are no allele in common, we try
       to flip one. If the new sets have one allele in common, we set the
       status to ``homo_hetero_flip``.
    5. If the sets of available alleles are the same (without flip), we check
       the frequency and the minor alleles. If the minor allele is different,
       we set the status to ``diff_minor_allele``. If the difference in
       frequencies is higher than a threshold, we set the status to
       ``diff_frequency``.
    6. If all of the above fail, we set the status to ``problem``.

    Problems are written in the ``prefix.problems`` file, and contains the
    following columns: chromosome, position, name and status. This file
    contains all the markers with a status, as explained above.

def printProblems(completion, concordance, tped, snps, frequencies, prefix,
                  diffFreq):
    """Print the statistics.

    :param completion: the completion of each duplicated markers.
    :param concordance: the pairwise concordance between duplicated markers.
    :param tped: a representation of the ``tped`` of duplicated markers.
    :param snps: the positions of the duplicated markers in the ``tped``
    :param frequencies: the frequency of each of the duplicated markers.
    :param prefix: the prefix of the output files.
    :param diffFreq: the frequency difference threshold.

    :type completion: numpy.array
    :type concordance: dict
    :type tped: numpy.array
    :type snps: dict
    :type frequencies: dict
    :type prefix: str
    :type diffFreq: float

    :returns: a :py:class:`set` containing duplicated markers to complete.

    Creates a summary file (``prefix.summary``) containing information about
    duplicated markers: chromosome, position, name, alleles, status, completion
    percentage, completion number and mean concordance.

    The frequency and the minor allele are used to be certain that two
    duplicated markers are exactly the same marker (and not a tri-allelic one,
    for example).

    For each duplicated markers:

    1. Constructs the set of available alleles for the first marker.
    2. Constructs the set of available alleles for the second marker.
    3. If the two sets are different, but the number of alleles is the same, we
       try to flip one of the marker. If the two sets are the same, but the
       number of alleles is 1, we set the status to ``homo_flip``. If the
       markers are heterozygous, we set the status to ``flip``.
    4. If there is a difference in the number of alleles (one is homozygous,
       the other, heterozygous), and that there is on allele in common, we set
       the status to ``homo_hetero``. If there are no allele in common, we try
       to flip one. If the new sets have one allele in common, we set the
       status to ``homo_hetero_flip``.
    5. If the sets of available alleles are the same (without flip), we check
       the frequency and the minor alleles. If the minor allele is different,
       we set the status to ``diff_minor_allele``. If the difference in
       frequencies is higher than a threshold, we set the status to
       ``diff_frequency``.
    6. If all of the above fail, we set the status to ``problem``.

    Problems are written in the ``prefix.problems`` file, and contains the
    following columns: chromosome, position, name and status. This file
    contains all the markers with a status, as explained above.

    """

    completionPercentage = np.true_divide(completion[0], completion[1])

    outSummary = None
    try:
        outSummary = open(prefix + ".summary", "w")
    except IOError:
        msg = "%s: can't write file" % prefix + ".summary"
        raise ProgramError

    # Prints the header of the summary file
    print >>outSummary, "\t".join(["chr", "pos", "name", "alleles", "status",
                                   "% completion", "completion",
                                   "mean concordance"])

    # The data structure containing the problems
    problems = {}
    for snpID, indexes in snps.iteritems():
        for i, index in enumerate(indexes):
            # The SNP information (chromosome and position)
            toPrint = list(snpID)

            # The name of the SNP
            snpName = tped[index, 1]
            toPrint.append(snpName)

            # The frequency of the SNP
            snpFreq, mafAlleles = frequencies[snpName]

            # A list of the other SNP name with problems
            otherSnpNameWithProblem = set()

            # The alleles
            alleles = set()
            otherAlleles = set()
            status = []
            for genotype in np.unique(tped[index, 4:]):
                alleles |= set(genotype.split(" "))
            if "0" in alleles:
                alleles.remove("0")
            for j in xrange(i+1, len(indexes)):
                otherIndex = indexes[j]
                otherSnpName = tped[otherIndex, 1]

                # The frequency of the other SNP
                otherSnpFreq, otherMafAlleles = frequencies[otherSnpName]

                # Checking the alleles
                for genotype in np.unique(tped[otherIndex, 4:]):
                    otherAlleles |= set(genotype.split(" "))
                if "0" in otherAlleles:
                    otherAlleles.remove("0")
                if alleles != otherAlleles:
                    if len(alleles) == len(otherAlleles):
                        # Same number of alleles
                        # Try the flipped ones
                        otherAlleles = flipGenotype(otherAlleles)
                        if alleles == otherAlleles:
                            if len(alleles) == 1:
                                status.append("homo_flip")
                                otherSnpNameWithProblem.add(otherSnpName)
                            else:
                                status.append("flip")
                                otherSnpNameWithProblem.add(otherSnpName)
                        else:
                            status.append("problem")
                            otherSnpNameWithProblem.add(otherSnpName)
                    else:
                        # Different number of alleles
                        if len(alleles & otherAlleles) == 1:
                            status.append("homo_hetero")
                            otherSnpNameWithProblem.add(otherSnpName)
                        else:
                            # Try the flipped one
                            otherAlleles = flipGenotype(otherAlleles)
                            if len(alleles & otherAlleles) == 1:
                                status.append("homo_hetero_flip")
                                otherSnpNameWithProblem.add(otherSnpName)
                            else:
                                status.append("problem")
                                otherSnpNameWithProblem.add(otherSnpName)
                else:
                    # The alleles are the same, so we check the frequency
                    if mafAlleles[0] != otherMafAlleles[0]:
                        # They don't have the same minor allele
                        status.append("diff_minor_allele")
                        otherSnpNameWithProblem.add(otherSnpName)
                    elif math.fabs(snpFreq - otherSnpFreq) > diffFreq:
                        # They don't have same frequency
                        status.append("diff_frequency")
                        otherSnpNameWithProblem.add(otherSnpName)

            alleles = list(alleles)
            alleles.sort()
            if len(alleles) == 1:
                alleles.append(alleles[0])
            toPrint.append(" ".join(alleles))
            toPrint.append(";".join(status))

            # The completion
            toPrint.append("%.8f" % completionPercentage[index])
            toPrint.append("%d/%d" % (completion[0][index],
                                      completion[1][index]))

            # The concordance
            indexToKeep = list(set(range(len(indexes))) - set([i]))
            currConcordance = np.true_divide(
                concordance[snpID][0][i, indexToKeep],
                concordance[snpID][1][i, indexToKeep],
            )
            currConcordance = np.mean(currConcordance)
            toPrint.append("%.8f" % currConcordance)
            print >>outSummary, "\t".join(toPrint)

            # Now updating the problems data structure
            if len(status) != len(otherSnpNameWithProblem):
                msg = "There is a problem with the problematic SNPs"
                raise ProgramError(msg)

            if len(status) > 0:
                if snpID not in problems:
                    tmp = {"snpNames": {snpName}, "problems": set()}
                    problems[snpID] = tmp

                # We have problems
                problems[snpID]["snpNames"] |= otherSnpNameWithProblem
                problems[snpID]["problems"] |= set(status)

    outSummary.close()

    outProblems = None
    try:
        outProblems = open(prefix + ".problems", "w")
    except IOError:
        msg = "%s: can't write file" % prefix + ".problems"
        raise ProgramError

    # Printing the header of the problem file...
    print >>outProblems, "\t".join(["chr", "pos", "name", "status"])
    for snpID in problems.iterkeys():
        toPrint = list(snpID)
        toPrint.append(";".join(list(problems[snpID]["snpNames"])))
        toPrint.append(";".join(list(problems[snpID]["problems"])))
        print >>outProblems, "\t".join(toPrint)

    outProblems.close()

    # Returning the SNPs to complete
    return set(snps.keys()) - set(problems.keys())