- Although author state that components can be replaced by other models for flexibility, authors did not try any change or alternative in the paper to proof the robustness of the proposed framework.
- Did authors tried using BlenderBot vs 2.0 with incorporated knowledge? it would be very interesting to see how the dialogs can be improved by using domain ontologies from the SGD dataset. - Although BlenderBot is finetuned on the SGD dataset, it is not clear how using more specific TOD chatbots can provide better results - Lines 159-162: Authors should provide more information about the type/number of personas created, and how the personas are used by the chatbot to generate the given responses. - It is not clear if authors also experimented with the usage of domain ontologies to avoid the generation of placeholders in the evaluated responses - Line 211: How many questions were created for this zero-shot intent classifier and what is the accuracy of this system?
- Line 216: How many paraphrases were created for each question, and what was their quality rate?
- Line 237: How critical was the finetuning process over the SQuad and CommonsenseQA models?
- Line 254-257: How many templates were manually created? - Line 265: How the future utterances are used during evaluation? For the generation part, are the authors generating some sort of sentence embedding representation (similar to SkipThoughs) to learn the generation of the transition sentence? and is it the transition sentence one taken from the list of manual templates? ( In general, this section 2.2.2 is the one I have found less clear) - Merge SGD: Did authors select the TOD dialogue randomly from those containing the same intent/topic? did you tried some dialogue embedding from the ODD part and tried to select a TOD dialogue with a similar dialogue embedding? if not, this could be an idea to improve the quality of the dataset. this could also allow the usage of the lexicalized version of the SGD and avoids the generation of placeholders in the responses - Line 324: how the repeated dialogues are detected? - Line 356: how and how many sentences are finally selected from the 120 generated sentences?
- Lines 402-404: How the additional transitions are generated? using the T5 model? how many times the manual sentences were selected vs the paraphrased ones?
- The paper: Fusing task-oriented and open-domain dialogues in conversational agents is not included in the background section and it is important in the context of similar datasets - Probably the word salesman is misleading since by reading some of the generated dialogues in the appendixes, it is not clear that the salesman agent is in fact selling something. It seems sometimes that they are still doing chitchat but on a particular topic or asking for some action to be done (like one to be done by an intelligent speaker)
- It is not clear if authors also experimented with the usage of domain ontologies to avoid the generation of placeholders in the evaluated responses - Line 211: How many questions were created for this zero-shot intent classifier and what is the accuracy of this system?