(e.g., time, money, technology). For example, in medical datasets input features may correspond to lab tests, medical imaging results, or even answers to questionnaires, which are expensive and slow to produce. Allowing oneself to be able to accurately predict a response variable from a small set of input features is thus a desirable goal, which can be manifested in saving time, money, and sometimes even human lives.

As a running example, consider the case of a patient complaining to a family doctor about not feeling well. The doctor then asks the patient several questions about his current condition and medical background, and may also ask the patient to do some lab tests. Implicitly, the doctor is aiming at quickly collecting relevant details on the patient that will allow her to have a clear understanding of the patients’ medical status, and consequently decide on an appropriate action (e.g., medication prescription, admit to hospitalization etc.). The doctor would keep asking questions as long as this improves her understanding of the patient’s medical status. Whenever the information acquired so far enables her to obtain a clear picture of the patient’s status, she can make a decision about the next required steps. Making a 