Embodiments employ an inference method for neural networks that enforces deterministic constraints on outputs without performing post-processing or expensive discrete search over the feasible space. Instead, for each input, the continuous weights are nudged until the network's unconstrained inference procedure generates an output that satisfies the constraints. This is achieved by expressing the hard constraints as an optimization problem over the continuous weights and employing backpropagation to change the weights of the network. Embodiments optimize over the energy of the violating outputs; since the weights directly determine the output through the energy, embodiments are able to manipulate the unconstrained inference procedure to produce outputs that conform to global constraints.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method comprising: training a neural network, during a training phase of the neural network, to produce a trained neural network; after completing the training phase of the neural network: performing inference, using the trained neural network comprising a first set of weights, on a particular unlabeled input to produce a first output; determining whether the first output violates one or more constraints on output values of the trained neural network; in response to determining that the first output violates the one or more constraints on output values of the trained neural network, using backpropagation to adjust the first set of weights of the trained neural network to produce an adjusted trained neural network comprising a second set of weights; wherein the first set of weights is different than the second set of weights; performing inference, using the adjusted trained neural network, on the particular unlabeled input to produce a second output; wherein the second output is different from the first output; and wherein the method is performed by one or more computing devices. 2. The method of claim 1, wherein using backpropagation to adjust the first set of weights of the trained neural network to produce the adjusted trained neural network is performed during a testing phase of the neural network. 3. The method of claim 1, further comprising, after performing inference, using the adjusted trained neural network, on the particular unlabeled input to produce the second output: determining whether the second output violates the one or more constraints on output values of the trained neural network; in response to determining that the second output violates the one or more constraints on output values of the trained neural network, using backpropagation to adjust the second set of weights of the trained neural network to produce a second adjusted trained neural network comprising a third set of weights; wherein the third set of weights is different than both the first and second sets of weights; 4. The method of claim 1, further comprising, after performing inference, using the adjusted trained neural network, on the particular unlabeled input to produce the second output: determining whether the second output violates the one or more constraints; in response to determining that the second output does not violate the one or more constraints on output values of the trained neural network, storing the second output as an inference result of the particular unlabeled input. 5. The method of claim 4, wherein: the trained neural network, produced by said training the neural network, comprises an original set of weights; the method further comprises, after determining that the second output does not violate the one or more constraints on output values of the trained neural network, performing inference, using the trained neural network comprising the original set of weights, on a second unlabeled input. 6. The method of claim 1, wherein determining whether the first output violates one or more constraints on output values of the trained neural network is based on a loss function that encodes the one or more constraints. 7. The method of claim 6, further comprising: receiving a definition of the one or more constraints; and automatically formulating the loss function based on the definition of the one or more constraints. 8. The method of claim 7, wherein the definition of the one or more constraints is defined using one or more of: context-free language; or regular language. 9. The method of claim 1, wherein using backpropagation to adjust the first set of weights of the trained neural network to produce the adjusted trained neural network further comprises: applying stochastic gradient descent to compute a gradient of a negative energy function to determine a set of changes to be applied to the first set of weights; and applying the set of changes to the first set of weights to produce the second set of weights. 10. The method of claim 1, wherein the one or more constraints are hard constraints. 11. One or more non-transitory computer-readable media storing one or more sequences of instructions which, when executed by one or more processors, cause: training a neural network, during a training phase of the neural network, to produce a trained neural network; after completing the training phase of the neural network: performing inference, using the trained neural network comprising a first set of weights, on a particular unlabeled input to produce a first output; determining whether the first output violates one or more constraints on output values of the trained neural network; in response to determining that the first output violates the one or more constraints on output values of the trained neural network, using backpropagation to adjust the first set of weights of the trained neural network to produce an adjusted trained neural network comprising a second set of weights; wherein the first set of weights is different than the second set of weights; performing inference, using the adjusted trained neural network, on the particular unlabeled input to produce a second output; wherein the second output is different from the first output. 12. The one or more non-transitory computer-readable media of claim 11, wherein using backpropagation to adjust the first set of weights of the trained neural network to produce the adjusted trained neural network is performed during a testing phase of the neural network. 13. The one or more non-transitory computer-readable media of claim 11, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause, after performing inference, using the adjusted trained neural network, on the particular unlabeled input to produce the second output: determining whether the second output violates the one or more constraints on output values of the trained neural network; in response to determining that the second output violates the one or more constraints on output values of the trained neural network, using backpropagation to adjust the second set of weights of the trained neural network to produce a second adjusted trained neural network comprising a third set of weights; wherein the third set of weights is different than both the first and second sets of weights; 14. The one or more non-transitory computer-readable media of claim 11, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause, after performing inference, using the adjusted trained neural network, on the particular unlabeled input to produce the second output: determining whether the second output violates the one or more constraints; in response to determining that the second output does not violate the one or more constraints on output values of the trained neural network, storing the second output as an inference result of the particular unlabeled input. 15. The one or more non-transitory computer-readable media of claim 14, wherein: the trained neural network, produced by said training the neural network, comprises an original set of weights; the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause, after determining that the second output does not violate the one or more constraints on output values of the trained neural network, performing inference, using the trained neural network comprising the original set of weights, on a second unlabeled input. 16. The one or more non-transitory computer-readable media of claim 11, wherein determining whether the first output violates one or more constraints on output values of the trained neural network is based on a loss function that encodes the one or more constraints. 17. The one or more non-transitory computer-readable media of claim 16, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause: receiving a definition of the one or more constraints; and automatically formulating the loss function based on the definition of the one or more constraints. 18. The one or more non-transitory computer-readable media of claim 17, wherein the definition of the one or more constraints is defined using one or more of: context-free language; or regular language. 19. The one or more non-transitory computer-readable media of claim 11, wherein using backpropagation to adjust the first set of weights of the trained neural network to produce the adjusted trained neural network further comprises: applying stochastic gradient descent to compute a gradient of a negative energy function to determine a set of changes to be applied to the first set of weights; and applying the set of changes to the first set of weights to produce the second set of weights. 20. The one or more non-transitory computer-readable media of claim 11, wherein the one or more constraints are hard constraints.