Access control is a fundamental aspect of security. There are many variations of the basic access control models and it is confusing for a software developer to select an appropriate model for her application. The result in practice is that only basic models are used and the power of more advanced models is thus lost. We try to clarify this panorama here through the use of patterns. In particular, we use pattern diagrams to navigate the pattern space. A pattern diagram shows relationships between patterns and we can see how different models relate to each other. A subproduct of our work is the analysis of which patterns are available for use and which need to be written. Pattern maps are also useful to perform semi-automatic model transformations as required for Model-Driven Development (MDD). The idea is to provide the designer of a secure system with a navigation tool that she can use to select an appropriate pattern from a catalog of security patterns. We also indicate how to compose new access control models by adding features to an existing pattern and how to define patterns by analogy.