The present disclosure relates to the electronic document review field and, more particularly, to various apparatuses and methods of implementing batch-mode active learning for technology-assisted review (TAR) of documents (e.g., legal documents).
Please help me write a proper abstract based on the patent claims. CLAIM: 1. An apparatus configured to implement batch-mode active learning for technology-assisted review (TAR) of documents, the apparatus comprising: a processor; and, a memory that stores processor-executable instructions, wherein the processor interfaces with the memory to execute the processor-executable instructions, whereby the apparatus is operable to: obtain an unlabeled set of documents D; obtain a batch size k; construct a first batch of k documents D; obtain labels for the first batch of k documents D, wherein the labeled first batch of k documents D are referred to as training data documents; construct a hyperplane hc(x) using the labeled first batch of k documents D; perform an iteration of active learning using a support vector machine (SVM), wherein the perform operation comprises: (i) select a new batch of unlabeled instances Bc using a current version of the hyperplane hc(x), an unlabeled set of available documents D, and the batch size k, wherein the apparatus is further operable to implement a Diversity Sampler process or a Biased Probabilistic Sampler process to select the new batch of unlabeled instances Bc; (ii) obtain labels for the new batch of unlabeled instances Bc; (iii) add the labeled new batch of instances Bc to a current version of the training data documents referred to as extended training data documents Dc; construct an updated hyperplane h(x) using the extended training data documents Dc; determine whether a stopping criteria has been met; based on the determination that the stopping criteria has not been met, repeat the perform operation, the third construct operation, and the determine operation; and, based on the determination that the stopping criteria has been met, return the updated hyperplane h(x). 2. The apparatus of claim 1, wherein the apparatus is further operable to implement the Diversity Sampler process as follows: obtain the current version of the hyperplane hc(x), the unlabeled set of available documents D, the batch size k, and a cosine similarity threshold t; sort the unlabeled set of available documents D in increasing order based on each of the documents absolute distance from the current version of the hyperplane hc(x) to obtain sorted indices I of the unlabeled set of available documents D; insert the sorted document having a nearest sorted indice I[1] of the sorted unlabeled set of available documents D into the new batch of unlabeled instances Bc; obtain sorted indices I of the sorted unlabeled set of available documents D that have a cosine angle ≧t with respect to the inserted document I[1]; remove the documents with the obtained sorted indices I from the sorted unlabeled set of available documents D; and repeat the insert operation, the second obtain operation, and the remove operation until insert k documents into the new batch of unlabeled instances Bc. 3. The apparatus of claim 1, wherein the apparatus is further operable to implement the Biased Probabilistic Sampler process as follows: obtain the current version of the hyperplane hc(x), the unlabeled set of available documents D, the batch size k, and a cosine similarity threshold t; construct a weight vector w based on an inverse operation of a distance from the current version of the hyperplane hc(x) for each document from the unlabeled set of available documents D; normalize the weight vector w to convert the weight vector w into a probability vector; choose a document I[1] from the unlabeled set of available documents D using the weight w of the probability vector; insert the chosen document I[1] into the new batch of unlabeled instances Bc; obtain sorted indices I of the remaining unlabeled set of available documents D that have a cosine angle ≧t with respect to the chosen document I[1]; remove the documents with the obtained sorted indices I from the unlabeled set of available documents D; re-normalize the weight vector as documents have been removed from the sorted unlabeled set of available documents D; and, repeat the choose operation, the insert operation, the second obtain operation, the remove operation, and the re-normalize operation until select the new batch of unlabeled instances Bc. 4. The apparatus of claim 1, wherein the apparatus is further operable to determine whether the stopping criteria has been met by implementing a Kappa agreement which is a measure of how much the current version of the hyperplane hc(x) and the updated hyperplane h(x) agree on respective prediction of labels of a chosen set of the documents D. 5. The apparatus of claim 1, wherein the apparatus is further operable to determine whether the stopping criteria has been met by implementing an Accumulated History of Scores process which comprises the following operations: obtain the current version of the hyperplane h(x), the unlabeled set of available documents D and the extended training data documents Dc referred to as the total set of documents DT, an accumulation function A(Sc, Sc−1, . . . , S1), and a stopping threshold tstop; construct a score vector Sc using the current hyperplane h(x) and the unlabeled set of documents D; combine a current score vector Sc with previous score vectors (Sc−1, . . . , S1) using the accumulation function A(Sc, Sc−1, . . . , S1) to obtain a stability value (s); compare the stability value s to the stopping threshold tstop to determine whether tstop≦S which indicates that the stopping criteria has been met; and based on the determination that the stopping criteria has not been met, store the current score vector Sc to memory as a previous score vector Sc−1. 6. The apparatus of claim 1, wherein the apparatus is further operable to determine whether the stopping criteria has been met by implementing a Cohen's Kappa process which comprises the following operations: obtain the current version of the hyperplane h(x), the unlabeled set of available documents D and the extended training data documents Dc referred to as the total set of documents DT, and a stopping threshold tstop; construct a score vector Sc using the current hyperplane h(x) and the total set of documents DT; retrieve a previous score vector Sc−1; obtain a set of documents D+c from the documents DT that have a positive score in Sc; obtain a set of documents D−c from the documents DT that have a negative score in Sc obtain a set of documents D+(c−1) from the documents DT that have a positive score in Sc−1; obtain a set of documents D−(c−1) from the documents DT that have a negative score in Sc−1; obtain a set of documents in common, D+, between D+c and D+(c−1); obtain a set of documents in common, D−, between D−c and D−(c−1); obtain a probability, P+, of a document having positive score in both score vectors Sc and Sc−1 by counting a number of documents, N+, in D+ divided by the total number of documents, N, in D; obtain a probability, P−, of a document having negative score in both score vectors Sc and Sc−1, by counting a number of documents, N−, in D− divided by the total number of documents, N, in D; obtain a value Ao as P++P−; obtain a probability, P+c, by counting a number of documents, N+c, in D+c divided by the number of documents, N, in D; obtain a probability, P−c, by counting a number of documents, N−c, in D−c divided by the number of documents, N, in D; obtain a probability, P+(c−1), by counting a number of documents, N+(c−1), in D+(c−1) divided by the number of documents, N, in D; obtain a probability, P−(c−1), by counting a number of documents, N−(c−1), in D−(c−1) divided by the number of documents, N, in D; obtain a value Ae as a probability of obtaining a positive document, P+c*P+(c−1), plus a probability of obtaining a negative document, P−c*P−(c−1); obtain a Kappa value L as Ao−Ae divided by (1−Ao); compare the Kappa value L to the stopping threshold tstop to determine whether tstop≦Ao−Ae/1−Ae which indicates that the stopping criteria has been met; and, based on the determination that the stopping criteria has not been met, store the score vector Sc to memory as a previous score vector Sc−1. 7. The apparatus of claim 1, wherein the apparatus is further operable to: discriminate, using the updated hyperplane h(x), remaining documents from the unlabeled set of available documents D where each of the remaining documents are classified as relevant or non-relevant. 8. A method in an apparatus for implementing batch-mode active learning for technology-assisted review (TAR) of documents, the method comprising: obtaining an unlabeled set of documents D; obtaining a batch size k; constructing a first batch of k documents D; obtaining labels for the first batch of k documents D, wherein the labeled first batch of k documents D are referred to as training data documents; constructing a hyperplane hc(x) using the labeled first batch of k documents D; performing an iteration of active learning using a support vector machine (SVM), wherein the perform operation comprises: (i) selecting a new batch of unlabeled instances Bc using a current version of the hyperplane hc(x), an unlabeled set of available documents D, and the batch size k, wherein the apparatus is further operable to implement a Diversity Sampler process or a Biased Probabilistic Sampler process to select the new batch of unlabeled instances Bc; (ii) obtaining labels for the new batch of unlabeled instances Bc; (iii) adding the labeled new batch of instances Bc to a current version of the training data documents referred to as extended training data documents Dc; constructing an updated hyperplane h(x) using the extended training data documents Dc; determining whether a stopping criteria has been met; based on the determination that the stopping criteria has not been met, repeating the performing step, the third constructing step, and the determining step; and, based on the determination that the stopping criteria has been met, returning the updated hyperplane h(x). 9. The method of claim 8, wherein the Diversity Sampler process comprises following steps: obtaining the current version of the hyperplane hc(x), the unlabeled set of available documents D, the batch size k, and a cosine similarity threshold t; sorting the unlabeled set of available documents D in increasing order based on each of the documents absolute distance from the current version of the hyperplane hc(x) to obtain sorted indices I of the unlabeled set of available documents D; inserting the sorted document having a nearest sorted indice I[1] of the sorted unlabeled set of available documents D into the new batch of unlabeled instances Bc; obtaining sorted indices I of the sorted unlabeled set of available documents D that have a cosine angle ≧t with respect to the inserted document I[1]; removing the documents with the obtained sorted indices I from the sorted unlabeled set of available documents D; and repeating the insert operation, the second obtain operation, and the remove operation until insert k documents into the new batch of unlabeled instances Bc. 10. The method of claim 8, wherein the Biased Probabilistic Sampler process comprises the following steps: obtaining the current version of the hyperplane hc(x), the unlabeled set of available documents D, the batch size k, and a cosine similarity threshold t; constructing a weight vector w based on an inverse operation of a distance from the current version of the hyperplane hc(x) for each document from the unlabeled set of available documents D; normalizing the weight vector w to convert the weight vector w into a probability vector; choosing a document I[1] from the unlabeled set of available documents D using the weight w of the probability vector; inserting the chosen document I[1] into the new batch of unlabeled instances Bc; obtaining sorted indices I of the remaining unlabeled set of available documents D that have a cosine angle ≧t with respect to the chosen document I[1]; removing the documents with the obtained sorted indices I from the unlabeled set of available documents D; re-normalizing the weight vector as documents have been removed from the sorted unlabeled set of available documents D; and, repeating the choosing step, the inserting step, the second obtaining step, the removing step, and the re-normalizing step until select the new batch of unlabeled instances Bc. 11. The method of claim 8, wherein the step of determining whether the stopping criteria has been met further comprises implementing a Kappa agreement which is a measure of how much the current version of the hyperplane hc(x) and the updated hyperplane h(x) agree on respective prediction of labels of a chosen set of the documents D. 12. The apparatus of claim 8, wherein the step of determining whether the stopping criteria has been met further comprises implementing an Accumulated History of Scores process which comprises the following steps: obtaining the current version of the hyperplane h(x), the unlabeled set of available documents D and the extended training data documents Dc referred to as the total set of documents DT, an accumulation function A(Sc, Sc−1, . . . , S1), and a stopping threshold tstop; constructing a score vector Sc using the current hyperplane h(x) and the total set of documents DT; combining a current score vector Sc with previous score vectors (Sc−1, . . . , S1) using the accumulation function A(Sc, Sc−1, . . . , S1) to obtain a stability value(s); comparing the stability value s to the stopping threshold tstop to determine whether tstop≦S which indicates that the stopping criteria has been met; and based on the determination that the stopping criteria has not been met, storing the current score vector Sc to memory as a previous score vector Sc−1. 13. The method of claim 8, wherein the step of determining whether the stopping criteria has been met further comprises implementing a Cohen's Kappa process which comprises the following steps: obtaining the current version of the hyperplane h(x), the unlabeled set of available documents D and the extended training data documents Dc referred to as the total set of documents DT, and a stopping threshold tstop; constructing a score vector Sc using the current hyperplane h(x) and the total set of documents DT; retrieving a previous score vector Sc−1; obtaining a set of documents D+c from the documents DT that have a positive score in Sc; obtaining a set of documents D−c from the documents DT that have a negative score in Sc obtaining a set of documents D−(c−1) from the documents DT that have a positive score in Sc−1; obtaining a set of documents D−(c−1) from the documents DT that have a negative score in Sc−1; obtaining a set of documents in common, D+, between D+c and D+(c−1); obtaining a set of documents in common, D−, between D−c and D−(c−1); obtaining a probability, P+, of a document having positive score in both score vectors Sc and Sc−1 by counting a number of documents, N+, in D+ divided by the total number of documents, N, in D; obtaining a probability, P−, of a document having negative score in both score vectors Sc and Sc−1, by counting a number of documents, N−, in D− divided by the total number of documents, N, in D; obtaining a value Ao as P++P−; obtaining a probability, P+c, by counting a number of documents, N+c, in D+c divided by the number of documents, N, in D; obtaining a probability, P−c, by counting a number of documents, N−c, in D−c divided by the number of documents, N, in D; obtaining a probability, P+(c−1), by counting a number of documents, N+(c−1), in D+(c−1) divided by the number of documents, N, in D; obtaining a probability, P−(c−1), by counting a number of documents, N−(c−1), in D−(c−1) divided by the number of documents, N, in D; obtaining a value Ae as a probability of obtaining a positive document, P+c*P+(c−1), plus a probability of obtaining a negative document, P−c*P−(c−1); obtaining a Kappa value L as Ao−Ae divided by (1−Ao); comparing the Kappa value L to the stopping threshold tstop to determine whether tstop≦Ao−Ae/1−Ae which indicates that the stopping criteria has been met; and, based on the determination that the stopping criteria has not been met, storing the score vector Sc to memory as a previous score vector Sc−1. 14. The method of claim 8, further comprising: discriminating, using the updated hyperplane h(x), remaining documents from the unlabeled set of available documents D where each of the remaining documents are classified as relevant or non-relevant.