- The paper is hard to read and somewhat difficult to follow.
- The motivation is unclear. The authors argue that the LLP setup is relevant for (1) privacy and (2) weak supervision. (1) Privacy: the authors claim that the LLP paradigm is relevant for training on sensitive data as the labels for such datasets are not publicly available. However, the setting proposed in this paper does require gold (and publicly available) labels to formulate the ground truth proportion. If this proportion can be formulated without gold labels, it should be discussed. (2) Weak Supervision: in lines 136-137, the authors mention that the associated label proportions "...provides the weak supervision for training the model". However, weak supervision is a paradigm in which data is automatically labeled with noisy labels using some heuristics and labeling functions. It remains unclear to me in what way this setting is related to the proportion parameter authors use in their work.
- The authors claim it to be one of the preliminary works discussing the application of LLP to NLP tasks. However, I don't see anything NLP-specific in their approach.
- Not all theoretical groundings seem to be relevant to the main topic (e.g., some of the L_dppl irregularities). Additional clarification of their relevance is needed.
- Section 3.3 says the results are provided for binary classifiers only, and the multi-class setting remains for future work. However, one of the datasets used for experiments is multi-label.
- The experimental setting is unclear: does Table 1 contain the test results of the best model? If so, how was the best model selected (given that there is no validation set)? Also, if the proposed method is of special relevance to the sensitive data, why not select a sensitive dataset to demonstrate the method's performance on it? Or LLP data?
- The authors claim the results to be significant. However, no results of significance testing are provided.
- The authors claim it to be one of the preliminary works discussing the application of LLP to NLP tasks. However, I don't see anything NLP-specific in their approach.