discovered, providing an elegant and effective solution to the problem of association mining discovering large numbers of misleading and redundant rules.'
author:
- 'Wilhelmiina Hämäläinen$^1$\'
- |
    Geoffrey I. Webb$^2$\
    $^1$Department of Computer Science, Aalto University, Finland – wilhelmiina.hamalainen@aalto.fi\
    $^2$Faculty of IT, Monash University, Melbourne, Australia – geoff.webb@monash.edu
title: 'Specious rules: an efficient and effective unifying method for removing misleading and uninformative patterns in association rule mining'
---

*Keywords*: specious dependency, association rule, Yule-Simpson’s paradox, mutual information, Birch’s test

Introduction
============

Association rule mining is often a useful first step for any data mining task, because it reveals the statistical dependence structure of data. A major issue in association rule mining is that analyses often produce excessive numbers of rules. Even if rules are carefully pruned with strict statistical measures, the results are likely to contain large numbers of redundant, unproductive, uninteresting or even misleading and paradoxical rules. In this paper, we investigate the problem of [*specious associations*]{}, also known as spurious, apparent, illusory, or misleading associations (e.g., [@yule1903]). Specious associations refer to a well-known phenomenon where marginal dependencies appear as products of interactions with other variables and disappear when conditioned on those variables. The most extreme example is Yule-Simpson’s 