src/main/java/org/apache/sysds/runtime/compress/cocode/PlanningCoCoder.java
/**
	 * Main entry point of CoCode.
	 * 
	 * This package groups together ColGroups across columns, to improve compression further,
	 * 
	 * @param sizeEstimator The size estimator used for estimating ColGroups potential sizes.
	 * @param colInfos      The information already gathered on the individual ColGroups of columns.
	 * @param numRows       The number of rows in the input matrix.
	 * @param k             The concurrency degree allowed for this operation.
	 * @param cs            The Compression Settings used in the compression.
	 * @return The Estimated (hopefully) best groups of ColGroups.
	 */
Main entry point of CoCode.

This package groups together ColGroups across columns, to improve compression further.

@param sizeEstimator The size estimator used for estimating ColGroups potential sizes.
@param colInfos      The information already gathered on the individual ColGroups of columns.
@param numRows       The number of rows in the input matrix.
@param k             The concurrency degree allowed for this operation.
@param cs            The Compression Settings used in the compression.
@return The Estimated (hopefully) best groups of ColGroups.
Main entry point of CoCode . This package groups together ColGroups across columns to improve compression further . @param sizeEstimator The size estimator used for estimating ColGroups potential sizes . @param colInfos The information already gathered on the individual ColGroups of columns . @param numRows The number of rows in the input matrix . @param k The concurrency degree allowed for this operation . @param cs The Compression Settings used in the compression . @return The Estimated ( hopefully ) best groups of ColGroups .
public static List<int[]> findCoCodesByPartitioning(CompressedSizeEstimator sizeEstimator,
		CompressedSizeInfo colInfos, int numRows, int k, CompressionSettings cs) {
		List<Integer> cols = colInfos.colsC;
		CompressedSizeInfoColGroup[] colGroups = colInfos.compressionInfo;
		int numCols = cols.size();
		List<Integer> groupCols = new ArrayList<>();
		HashMap<Integer, GroupableColInfo> groupColsInfo = new HashMap<>();
		for(int i = 0; i < numCols; i++) {
			int colIx = cols.get(i);
			int cardinality = colGroups[colIx].getEstCard();
			double weight = ((double)cardinality) / numRows;
			groupCols.add(colIx);
			groupColsInfo.put(colIx, new GroupableColInfo(weight, colGroups[colIx].getMinSize(), cardinality));
		}
		List<int[]> bins = createColumnGroupPartitioner(cs.columnPartitioner)
			.partitionColumns(groupCols, groupColsInfo, cs);
		if (cs.columnPartitioner == PartitionerType.COST){
			return bins;
		}
		return (k > 1) ? getCocodingGroupsBruteForce(bins,
			groupColsInfo,
			sizeEstimator,
			numRows,
			k) : getCocodingGroupsBruteForce(bins, groupColsInfo, sizeEstimator, numRows);
	}
public static List < int [ ] > findCoCodesByPartitioning ( CompressedSizeEstimator sizeEstimator , CompressedSizeInfo colInfos , int numRows , int k , CompressionSettings cs ) { List < Integer > cols = colInfos . colsC ; CompressedSizeInfoColGroup [ ] colGroups = colInfos . compressionInfo ; int numCols = cols . size ( ) ; List < Integer > groupCols = new ArrayList < > ( ) ; HashMap < Integer , GroupableColInfo > groupColsInfo = new HashMap < > ( ) ; for ( int i = 0 ; i < numCols ; i ++ ) { int colIx = cols . get ( i ) ; int cardinality = colGroups [ colIx ] . getEstCard ( ) ; double weight = ( ( double ) cardinality ) / numRows ; groupCols . add ( colIx ) ; groupColsInfo . put ( colIx , new GroupableColInfo ( weight , colGroups [ colIx ] . getMinSize ( ) , cardinality ) ) ; } List < int [ ] > bins = createColumnGroupPartitioner ( cs . columnPartitioner ) . partitionColumns ( groupCols , groupColsInfo , cs ) ; if ( cs . columnPartitioner == PartitionerType . COST ) { return bins ; } return ( k > 1 ) ? getCocodingGroupsBruteForce ( bins , groupColsInfo , sizeEstimator , numRows , k ) : getCocodingGroupsBruteForce ( bins , groupColsInfo , sizeEstimator , numRows ) ; }
Main entry point of CoCode.
Main entry point of CoCode .
// filtering out non-group-able columns as singleton groups // weight is the ratio of its cardinality to the number of rows // use column group partitioner to create partitions of columns // brute force grouping within each partition