. In order to have standing to bring a derivative suit, a plaintiff must have been a shareholder at the time the cause of action arose or received the shares by operation of law from someone who held them at that time. § 800(b)(1). However, the court may in its discretion allow a suit to continue if the plaintiff does not meet the standing requirement if there is strong evidence supporting the claim, no other similar action is pending or likely to be instituted, the plaintiff acquired his or her shares before the facts supporting the claim were made public, the defendant will receive a benefit resulting from his breach of duty unless the suit is instituted, and neither the corporation or complaining shareholder will be unjustly enriched by the requested relief. Id. Additionally, the plaintiff must allege in his or her complaint the actions taken to obtain relief from the corporation or the reasons why no such actions were taken. Id.§ 800(b)(2).
Within thirty days of service of the complaint, the defendant or the corporation itself may move for a court order requiring the plaintiff to post a bond as security if there is no reasonable possibility the suit will benefit the corporation or if the defendant, if other than the corporation, did not participate in the transaction that is the subject of the suit. § 800(c). The maximum amount of the bond is $50,000 and the suit must be dismissed if a bond is ordered by the court and the plaintiff does not comply. § 800(e).
Although there is no direct shareholder recovery in a derivative suit on behalf of the corporation, a successful plaintiff may recover expenses for attorneys’ fees if the suit substantially benefits a distinct class of shareholders “among whom the costs of litigation can be fairly spread to prevent the unjust enrichment of class members at the expense of the successful litigant.” Cziraki v. Thunder Cats, Inc., 3 Cal. Rptr.3d 419, 423 (Cal. Ct. App. 2003).A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents

The amount of dialogue history to include in a conversational agent is often underestimated and/or set in an empirical and thus possibly naive way. This suggests that principled investigations into optimal context windows are urgently needed given that the amount of dialogue history and corresponding representations can play an important role in the overall performance of a conversational system. This paper studies the amount of history required by conversational agents for reliably predicting dialogue rewards. The task of dialogue reward prediction is chosen for investigating the effects of varying amounts of dialogue history and their impact on system performance. Experimental results using a dataset of 18K human-human dialogues report that lengthy dialogue histories of at least 10 sentences are preferred (25 sentences being the best in our experiments) over short ones, and that lengthy histories are useful for training dialogue reward predictors with strong positive correlations between target dialogue rewards and predicted ones.


Introduction
Deep Reinforcement Learning holds a lot of promise for training intelligent conversational machines [2,4,3], which can infer their behaviour from trial and error via interaction with an environment in order to optimise a long term reward signal [19,9]. This machine learning paradigm is interesting and worth studying because it resembles human learning [16,12], and has wide application from lowlevel to high-level control using a single or multiple modalities. However, its successful application is not trivial due to interactions with real world environments (not easy to model), complex decisionmaking at different levels of granularity (not easy to infer or specify), and unkwown reward signals (also not easy to infer or specify), among others. In this paper we focus our attention to the latter only, the reward signals, in the context of conversational agents. Specifically, we focus on open-ended dialogue agents because they have received little attention so far in the literature-as described in Section 2. This type of agents are interesting and challenging at the same time because the amount of input features and their combinations are simply vast, making decision-making difficult. Even when some reward functions have been proposed by previous works [7,8,17] it is unclear what is the best reward function to use for training future conversational agents or chatbots.
Our study uses a dataset of human-human dialogues, which we extend with noisy dialogues by replacing human responses with randomly chosen responses from other dialogues. While non-noisy dialogues aim to show human-like and desirable outputs, the noisy ones aim to exemplify less desirable behaviour. Each dialogue is automatically labelled with a numerical reward according to the amount of noise (distorted sentences)-the automatic labels facilitate its application to other datasets. This extended dataset is used for predicting dialogue rewards of open-ended conversations. As part of our study, we propose a new reward function that is easy to implement and that strongly correlates with test human-human dialogues. The latter is indeed possible by using lengthy dialogue histories.

Related Work
So far little attention has been devoted to predicting numerical rewards of open-ended dialogues, which can be used to optimise the behaviour of dialogue agents such as chit-chat chatbots. By dialogue reward prediction we mean 'estimating the quality of human-computer or human-human dialogues', where quality can be measured for example via task success, user satisfaction or humanlikeness, among others. Most previous related works focus on task-oriented dialogue systems, which require labelled data typically collected by a previously developed dialogue system. For example, [18] used previously collected human-computer dialogues in the domain of restaurants for training an LSTM-based dialogue regressor for predicting rewards of the form R = 20 × 1 success − N, where a value of 20 is given for a successful dialogue minus the number of dialogue turns. Such a regressor-which can be turned into a binary classifier for predicting success or failure-was retrained online together with a dialogue policy using active learning. [13] also use LSTM-based predictors of dialogue success (or failure) in the programming domain (StackOverflow data) based on word-based features, where success is given to a correctly answered question-failure otherwise. Similarly, [14] explores other machine learning techniques for predicting dialogue success but from acoustic features instead of word-based features. [20] predict dialogue rewards of the form R = (iq − 1) × 5 − N, where iq is an interaction quality value derived from a multiclass classifier. While speech recognition features and dialogue actions are used as inputs, the outputs used for training were obtained from three human labellers that judged the quality of the interactions. The reward predictor induced from a dataset of bus information was used for training policies in the domains of restaurants, hotels and laptops.
The related studies above, motivated by the earlier work of [21] in predicting problematic dialogues, have focused on task-oriented dialogues that aim to achieve a goal such as finding restaurant/hotel/laptop information, call routing, etc., see summary in Table 1. There is however little related research on human-human or human-computer dialogues where there is no clear notion of a task or goal to achieve-notable exceptions are [17,8,7]. [17] train a neural multiclass classifier and linear regressor from human-chatbot dialogues from Amazon Mechanical Turk (AMT). In this study, AMT human evaluators are given dialogues and are asked to rate the quality of candidate responses according to the following categories (or labels): 1 indicates an innapropriate response or one that does not make sense, 3 indicates neutral, and 5 is a highly appropriate and interesting response. The classifier receives inputs with 1458 features (including lexical, syntactic and semantic information) and outputs a label out of the five possible labels. The regressor aims to learn a reward function for open-ended dialogues. It receives inputs with 23 features (also including lexical, syntactic and semantic information) and outputs a real value between 1 and 5. This regressor was used to train neural-based reinforcement learning dialogue policies for open-ended dialogues - [11] used a similar approach in a restricted set of domains. [8] train a binary classifier for predicting whether a dialogue is human-generated or machine generated. The outputs of this classifier, derived from a two-class Softmax function, are used as dialogue rewards in order to train a generator that aims for deceiving such a classifier within an adversarial framework. [7] produces rewards of the form r(a, [p i, q i ]) = λ 1 r 1 + λ 2 r 2 + λ 3 r 3, where a refers to the generated response, [p i, q i ] are the previous two dialogue turns, r 1 and r 3 refer to likelihood outputs of Seq2Seq models, r 2 refers to dialogue history similarity, and λ i are empirically defined weights (easy of answering: λ 1 = 0.25, information flow: λ 2 = 0.25, and semantic coherence: λ 3 = 0.5). While [17] reports a 0.40 Pearson correlation coefficient between predicted rewards and AMT human judgements, it is unclear whether or not the binary classifier proposed by [8] or heuristic function of [7] can produce rewards that correlate with human judgements.
The work presented in the remaining of the paper goes beyond previous works as follows: 1. we study dialogue reward prediction for open-ended dialogues using automatically labelled data. This is in contrast with [17,11] who use paid human evaluators, which results in an expensive and time consuming process because large amounts of dialogues require labelling; 2. we use contextually-richer and automatically derived input features to represent dialogue histories. This is in contrast with [17,11] who use a small set of carefully designed features and with [7,8,17] who use short dialogue histories for dialogue reward prediction; and 3. we investigate the impact of dialogue history length