{"action":"closed","issue":{"url":"https://api.github.com/repos/jemalloc/jemalloc/issues/518","repository_url":"https://api.github.com/repos/jemalloc/jemalloc","labels_url":"https://api.github.com/repos/jemalloc/jemalloc/issues/518/labels{/name}","comments_url":"https://api.github.com/repos/jemalloc/jemalloc/issues/518/comments","events_url":"https://api.github.com/repos/jemalloc/jemalloc/issues/518/events","html_url":"https://github.com/jemalloc/jemalloc/issues/518","id":193147678,"number":518,"title":"je_mallocx returnes memory that doesn't come from arena chunk_hooks","user":{"login":"Kkulakowski","id":11735823,"avatar_url":"https://avatars.githubusercontent.com/u/11735823?v=3","gravatar_id":"","url":"https://api.github.com/users/Kkulakowski","html_url":"https://github.com/Kkulakowski","followers_url":"https://api.github.com/users/Kkulakowski/followers","following_url":"https://api.github.com/users/Kkulakowski/following{/other_user}","gists_url":"https://api.github.com/users/Kkulakowski/gists{/gist_id}","starred_url":"https://api.github.com/users/Kkulakowski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kkulakowski/subscriptions","organizations_url":"https://api.github.com/users/Kkulakowski/orgs","repos_url":"https://api.github.com/users/Kkulakowski/repos","events_url":"https://api.github.com/users/Kkulakowski/events{/privacy}","received_events_url":"https://api.github.com/users/Kkulakowski/received_events","type":"User","site_admin":false},"labels":[{"id":59304344,"url":"https://api.github.com/repos/jemalloc/jemalloc/labels/question","name":"question","color":"cc317c","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-12-02T15:46:56Z","updated_at":"2016-12-02T15:59:57Z","closed_at":"2016-12-02T15:59:57Z","body":"When using arena with custom chunk_hooks I'm hitting a situation where memory returned from je_mallocx when requesting said arena comes from memory not allocated by my custom chunk_hooks.\r\n\r\nThis happens when before arena creation je_malloc/je_free were used.\r\n\r\nFollowing code reproduces this issue (tested with jemalloc 4.3.1). It will print message like \"Could not find allocation 3585. 0x7ffff7279000 in chunk register!\" if memory returned by je_mallocx does not comes from chunk allocated by chunk_hooks.\r\n```C\r\n#include <stdio.h>\r\n#include <jemalloc/jemalloc.h>\r\n#include <sys/mman.h>\r\n\r\n\r\n#define CHUNK_REGISTER_SIZE 1048576\r\nuintptr_t chunk_register_addr[CHUNK_REGISTER_SIZE];\r\nsize_t chunk_register_len[CHUNK_REGISTER_SIZE];\r\nsize_t chunk_register_n;\r\n\r\nvoid add_chunk(void* addr, size_t len) {\r\n    if(chunk_register_n >= CHUNK_REGISTER_SIZE) {\r\n        printf(\"Chunk register full\\n\");\r\n        exit(1);\r\n    }\r\n    chunk_register_addr[chunk_register_n] = (uintptr_t)addr;\r\n    chunk_register_len[chunk_register_n] = len;\r\n    chunk_register_n++;\r\n}\r\n\r\nbool is_from_registered_chunk(void* addr) {\r\n    uintptr_t addr_val = (uintptr_t)addr;\r\n    size_t i;\r\n    for(i=0; i<chunk_register_n; i++) {\r\n        if((addr_val >= chunk_register_addr[i]) &&\r\n           (addr_val < chunk_register_addr[i] + chunk_register_len[i])) {\r\n            return true;\r\n        }\r\n    }\r\n    return false;\r\n}\r\n\r\nvoid *chunk_alloc(void *chunk, size_t size, size_t alignment,\r\n                       bool *zero, bool *commit, unsigned arena_ind)\r\n{\r\n    int err;\r\n    void *addr = NULL;\r\n\r\n    addr = mmap(chunk, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);\r\n\r\n    if (addr != MAP_FAILED) {\r\n        if (chunk != NULL && addr != chunk) {\r\n            /* wrong place */\r\n            munmap(addr, size);\r\n            return NULL;\r\n        }\r\n\r\n        if ((uintptr_t)addr & (alignment-1)) {\r\n            /* wrong alignment - FIXME */\r\n            munmap(addr, size);\r\n            return NULL;\r\n        }\r\n\r\n        *zero = true;\r\n        *commit = true;\r\n    } else {\r\n        addr = NULL;\r\n    }\r\n\r\n    add_chunk(addr, size);\r\n    return addr;\r\n}\r\n\r\nbool chunk_dalloc(void *chunk, size_t size, bool commited,\r\n                        unsigned arena_ind)\r\n{\r\n    return true;\r\n}\r\n\r\nbool chunk_commit(void *chunk, size_t size, size_t offset, size_t length,\r\n                        unsigned arena_ind)\r\n{\r\n    return false;\r\n}\r\n\r\nbool chunk_decommit(void *chunk, size_t size, size_t offset, size_t length,\r\n                          unsigned arena_ind)\r\n{\r\n    return true;\r\n}\r\n\r\nbool chunk_purge(void *chunk, size_t size, size_t offset, size_t length,\r\n                       unsigned arena_ind)\r\n{\r\n    return false;\r\n}\r\n\r\nbool chunk_split(void *chunk, size_t size, size_t size_a, size_t size_b,\r\n                       bool commited, unsigned arena_ind)\r\n{\r\n    return false;\r\n}\r\n\r\nbool chunk_merge(void *chunk_a, size_t size_a, void *chunk_b,\r\n                       size_t size_b, bool commited, unsigned arena_ind)\r\n{\r\n    return true;\r\n}\r\n\r\nstatic chunk_hooks_t chunk_hooks = {\r\n    chunk_alloc,\r\n    chunk_dalloc,\r\n    chunk_commit,\r\n    chunk_decommit,\r\n    chunk_purge,\r\n    chunk_split,\r\n    chunk_merge\r\n};\r\n\r\nint main()\r\n{\r\n    int err;\r\n\r\n    //cause some memory preallocation with series of je_malloc() and je_free()\r\n    int i;\r\n    void* ptrs[100];\r\n    for(i=0; i<sizeof(ptrs)/sizeof(void*); i++)\r\n    {\r\n        ptrs[i] = je_malloc(4096);\r\n    }\r\n\r\n    for(i=0; i<sizeof(ptrs)/sizeof(void*); i++)\r\n    {\r\n        je_free(ptrs[i]);\r\n    }\r\n\r\n    unsigned arena_index;\r\n    size_t unsigned_size = sizeof(unsigned);\r\n    size_t chunk_hooks_t_size = sizeof(chunk_hooks_t);\r\n\r\n    //create new arena\r\n    err = je_mallctl(\"arenas.extend\", (void*)&arena_index, &unsigned_size, NULL, 0);\r\n    if(err) {\r\n        printf(\"mallctl error\\n\");\r\n        return err;\r\n    }\r\n\r\n    //setup chunk_hooks for newly created arena\r\n    char cmd[64];\r\n    snprintf(cmd, sizeof(cmd), \"arena.%u.chunk_hooks\", arena_index);\r\n    err = je_mallctl(cmd, NULL, NULL, (void*)&chunk_hooks, chunk_hooks_t_size);\r\n    if(err) {\r\n        printf(\"mallctl error\\n\");\r\n        return err;\r\n    }\r\n\r\n    size_t size;\r\n    for(size=1; size<4096; size++) {\r\n        //perform allocation using our custom arena, and make sure that memory comes from chunk_hook\r\n        void* ptr = je_mallocx(size, MALLOCX_ARENA(arena_index));\r\n        if(!is_from_registered_chunk(ptr)) {\r\n            printf(\"Could not find allocation %zu. %p in chunk register!\\n\", size, ptr);\r\n        }\r\n    }\r\n\r\n    return 0;\r\n}\r\n```\r\n\r\nBest Regards,\r\nKrzysztof"}}
{"actor":{"display_login":"jasone"}}