{"action":"created","issue":{"url":"https://api.github.com/repos/zheng-da/FlashX/issues/232","repository_url":"https://api.github.com/repos/zheng-da/FlashX","labels_url":"https://api.github.com/repos/zheng-da/FlashX/issues/232/labels{/name}","comments_url":"https://api.github.com/repos/zheng-da/FlashX/issues/232/comments","events_url":"https://api.github.com/repos/zheng-da/FlashX/issues/232/events","html_url":"https://github.com/zheng-da/FlashX/issues/232","id":162574012,"number":232,"title":"[Flash Graph] Preprocessing fails","user":{"login":"casusfortuna","id":1235951,"avatar_url":"https://avatars.githubusercontent.com/u/1235951?v=3","gravatar_id":"","url":"https://api.github.com/users/casusfortuna","html_url":"https://github.com/casusfortuna","followers_url":"https://api.github.com/users/casusfortuna/followers","following_url":"https://api.github.com/users/casusfortuna/following{/other_user}","gists_url":"https://api.github.com/users/casusfortuna/gists{/gist_id}","starred_url":"https://api.github.com/users/casusfortuna/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/casusfortuna/subscriptions","organizations_url":"https://api.github.com/users/casusfortuna/orgs","repos_url":"https://api.github.com/users/casusfortuna/repos","events_url":"https://api.github.com/users/casusfortuna/events{/privacy}","received_events_url":"https://api.github.com/users/casusfortuna/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-06-27T23:54:54Z","updated_at":"2016-06-29T18:15:43Z","closed_at":null,"body":"Hi, I would like to run several graph algorithms on a ~2billion(~20G) edge graph with FlashGraph. I have a server which has 16 cores, 32gb of memory 2*240GB SSD and 2TB HDD. However when I run el2fg tool to convert edge list into flashgraph format system fails. I changed configuration files as described in the tutorial and also I add another ssd and mount it as swap. However when I run el2fg memory usage exceeds 30G and virtual memory usage exceeds 80G and after a while system kills itself. Am I doing something wrong? Have you ever encountered with such a problem?"},"comment":{"url":"https://api.github.com/repos/zheng-da/FlashX/issues/comments/229442047","html_url":"https://github.com/zheng-da/FlashX/issues/232#issuecomment-229442047","issue_url":"https://api.github.com/repos/zheng-da/FlashX/issues/232","id":229442047,"user":{"login":"casusfortuna","id":1235951,"avatar_url":"https://avatars.githubusercontent.com/u/1235951?v=3","gravatar_id":"","url":"https://api.github.com/users/casusfortuna","html_url":"https://github.com/casusfortuna","followers_url":"https://api.github.com/users/casusfortuna/followers","following_url":"https://api.github.com/users/casusfortuna/following{/other_user}","gists_url":"https://api.github.com/users/casusfortuna/gists{/gist_id}","starred_url":"https://api.github.com/users/casusfortuna/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/casusfortuna/subscriptions","organizations_url":"https://api.github.com/users/casusfortuna/orgs","repos_url":"https://api.github.com/users/casusfortuna/repos","events_url":"https://api.github.com/users/casusfortuna/events{/privacy}","received_events_url":"https://api.github.com/users/casusfortuna/received_events","type":"User","site_admin":false},"created_at":"2016-06-29T18:15:43Z","updated_at":"2016-06-29T18:15:43Z","body":"Hello, thanks again for your help. My config file is as follows:\r\n`# Parameters for SAFS\r\n\r\n# How data blocks of a SAFS file are mapped to Linux filesystem.\r\nRAID_mapping=RAID0\r\n\r\n# The number of parallel I/Os is allowed in a single thread.\r\nio_depth=64\r\n\r\n# The page cache size in SAFS\r\ncache_size=512M\r\n\r\n# The number of NUMA nodes\r\nnum_nodes=1\r\n\r\n# change FG_TOP to the path of the top directory of FlashGraph.\r\nroot_conf=/home/ayasar/FlashGraph/flash-graph/conf/data_files.txt\r\n\r\n# whether or not to use Linux huge pages\r\n# huge_page=\r\n\r\n# The number of I/O threads per NUMA node.\r\nnum_io_threads=1\r\n\r\n\r\n# Parameters for FlashGraph\r\n\r\n# The number of worker threads to process a graph.\r\nthreads=12\r\n\r\n# The max number of vertices being processed in parallel in a worker thread\r\n# max_processing_vertices=1000\r\n\r\n# This defines the range size of a partition.\r\n# part_range_size_log=10\r\n\r\n# Keep adjacency lists of vertices in memory\r\n# in_mem_graph=\r\n\r\n# Run the vertex program on a vertx serially regardless of load balancing.\r\nserial_run=\r\n\r\n# The number of vertical partitions on the graph.\r\n# num_vertical_parts=1\r\n\r\n# The minimal degree that a vertex is partitioned.\r\n# min_vpart_degree=1\r\n`\r\nand my data file:\r\n\r\n`# change FG_TOP to the location of the top directory of FlashGraph.\r\n0:/home/ayasar/FlashGraph/flash-graph/data\r\n0:/home/ayasar/disk1\r\n0:/home/ayasar/disk2\r\n`"}}
{"actor":{"display_login":"casusfortuna"}}