Department of Computer Science Sun, 18 Aug 2019 09:08:48 GMT 2019-08-18T09:08:48Z Sparse Bayesian reinforcement learning Sparse Bayesian reinforcement learning Lee, Minwoo This dissertation presents knowledge acquisition and retention methods for efficient and robust learning. We propose a framework for learning and memorizing, and we examine how we can use the memory for efficient machine learning. Temporal difference (TD) learning is a core part of reinforcement learning, and it requires function approximation. However, with function approximation, the most popular TD methods such as TD(λ), SARSA, and Q-learning lose stability and diverge especially when the complexity of the problem grows and the sampling distribution is biased. The biased samples cause function approximators such as neural networks to respond quickly to the new data by losing what was previously learned. Systematically selecting a most significant experience, our proposed approach gradually stores the snapshot memory. The memorized snapshots prevent forgetting important samples and increase learning stability. Our sparse Bayesian learning model maintains the sparse snapshot memory for efficiency in computation and memory. The Bayesian model extends and improves TD learning by utilizing the state information in hyperparameters for smart decision of action selection and filtering insignificant experience to maintain sparsity of snapshots for efficiency. The obtained memory can be used to further improve learning. First, the placement of the snapshot memories with a radial basis function kernel located at peaks of the value function approximation surface leads to an efficient way to search a continuous action space for practical application with fine motor control. Second, the memory is a knowledge representation for transfer learning. Transfer learning is a paradigm for knowledge generalization of machine learning and reinforcement learning. Transfer learning shortens the time for machine learning training by using the knowledge gained from similar tasks. The dissertation examines a practice approach that transfers the snapshots from non-goal-directive random movements to goal-driven reinforcement learning tasks. Experiments are described that demonstrate the stability and efficiency of learning in 1) traditional benchmark problems and 2) the octopus arm control problem without limiting or discretizing the action space. 2017 Summer; Includes bibliographical references. Tutorial on Minimum Output Sum of Squared Error filter Tutorial on Minimum Output Sum of Squared Error filter Sidhu, Rumpal Kaur One of the major challenges in computer vision is visual tracking. Many different techniques have been developed for visual tracking over time. Visual tracking using correlation filters is faster than other techniques. A visual tracking technique using Optimized Correlation Output Filters (OCOF), Average of Synthetic Exact Filter (ASEF) and Minimum Output Sum of Squared Error (MOSSE) filters is presented in this tutorial. The MOSSE filter is a stable correlation filter which can be initialized on a single frame of a video. The MOSSE filter adapts with the changes in the appearance of the object while tracking. Tracking using the MOSSE filter is not dependent on changes in lighting, non-rigid transformations, pose, and scale. This work presents systematic steps towards building the filter and its application as a tracker in visual tracking. A number of experiments are performed to demonstrate the working of the tracker. 2016 Spring; Includes bibliographical references. Towards a secure and efficient search over encrypted cloud data Towards a secure and efficient search over encrypted cloud data Strizhov, Mikhail Cloud computing enables new types of services where the computational and network resources are available online through the Internet. One of the most popular services of cloud computing is data outsourcing. For reasons of cost and convenience, public as well as private organizations can now outsource their large amounts of data to the cloud and enjoy the benefits of remote storage and management. At the same time, confidentiality of remotely stored data on untrusted cloud server is a big concern. In order to reduce these concerns, sensitive data, such as, personal health records, emails, income tax and financial reports, are usually outsourced in encrypted form using well-known cryptographic techniques. Although encrypted data storage protects remote data from unauthorized access, it complicates some basic, yet essential data utilization services such as plaintext keyword search. A simple solution of downloading the data, decrypting and searching locally is clearly inefficient since storing data in the cloud is meaningless unless it can be easily searched and utilized. Thus, cloud services should enable efficient search on encrypted data to provide the benefits of a first-class cloud computing environment. This dissertation is concerned with developing novel searchable encryption techniques that allow the cloud server to perform multi-keyword ranked search as well as substring search incorporating position information. We present results that we have accomplished in this area, including a comprehensive evaluation of existing solutions and searchable encryption schemes for ranked search and substring position search. Includes bibliographical references.; 2016 Summer Quantitative economics of security : software vulnerabilities and data breaches Quantitative economics of security : software vulnerabilities and data breaches Algarni, Abdullah Mahdi Security vulnerabilities can represent enormous risks to society and business organizations. A large percentage of vulnerabilities in software are discovered by individuals external to the developing organization. These vulnerabilities are often exchanged for monetary rewards or a negotiated selling price, giving rise to vulnerability markets. Some of these markets are regulated, while some are unregulated. Many buyers in the unregulated markets include individuals, groups, or government organizations who intend to use the vulnerabilities for potential attacks. Vulnerabilities traded through such markets can cause great economic, organizational, and national security risks. Vulnerability markets can reduce risks if the vulnerabilities are acquitted and remedied by the software developers. Studying vulnerability markets and their related issues will provide an insight into their underlying mechanisms, which can be used to assess the risks and develop approaches for reducing and mitigating the potential risks to enhance the security against the data breaches. Some of the aspects of vulnerability—discovery, dissemination, and disclosure—have received some recent attention. However, the role of interaction among the vulnerability discoverers and vulnerability acquirers has not yet been adequately addressed. This dissertation suggests that a major fraction of discoverers, a majority in some cases, are unaffiliated with the software developers and thus are free to disseminate the vulnerabilities they discover in any way they like. As a result, multiple vulnerability markets have emerged. In recent vulnerability discovery literature, the vulnerability discoverers have remained anonymous. Although there has been an attempt to model the level of their efforts, information regarding their identities, modes of operation, and what they are doing with the discovered vulnerabilities has not been explored. Reports of buying and selling the vulnerabilities are now appearing in the press; however, the nature of the actual vulnerability markets needs to be analyzed. We have attempted to collect detailed information. We have identified the most prolific vulnerability discoverers throughout the past decade and examined their motivation and methods. A large percentage of these discoverers are located outside of the US. We have contacted several of the most prolific discoverers in order to collect firsthand information regarding their techniques, motivations, and involvement in the vulnerability markets. We examine why many of the discoverers appear to retire after a highly successful vulnerability-finding career. We found that the discoverers had enough experience and good reputation to work officially with a good salary in some well- known software development companies. Many security breaches have been reported in the past few years, impacting both large and small organizations. Such breaches may occur through the exploitation of system vulnerabilities. There has been considerable disagreement about the overall cost and probability of such breaches. No significant formal studies have yet addressed this issue of risk assessment, though some proprietary approaches for evaluating partial data breach costs and probabilities have been implemented. These approaches have not been formally evaluated or compared and have not been systematically optimized. This study proposes a consolidated approach for identifying key factors contributing to the breach cost by minimizing redundancy among the factors. Existing approaches have been evaluated using the data from some of the well-documented breaches. It is noted that the existing models yield widely different estimates. The reasons for this variation are examined and the need for better models is identified. A complete computational model for estimating the costs and probabilities of data breaches for a given organization has been developed. We consider both the fixed and variable costs and the economy of scale. Assessing the impact of data breaches will allow organizations to assess the risks due to potential breaches and to determine the optimal level of resources and effort needed for achieving target levels of security. Includes bibliographical references.; 2016 Summer