{"push_id":2367388063,"size":10,"distinct_size":10,"ref":"refs/heads/pack-objects-reduce-memory-footprint","head":"b9918a38e237f4944e489e4519d725d5a4739192","before":"53fc59cc1bfedca32e956d35e43a90dd46b86895","commits":[{"sha":"37c8447aab3e39862e260bd842ca99b0e7b38b8f","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: document holes in struct object_entry.h","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/37c8447aab3e39862e260bd842ca99b0e7b38b8f"},{"sha":"f995629f9e8224a2ea088cc9a343fe76af257c73","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: turn type and in_pack_type to bitfields\n\nThis saves 8 bytes in sizeof(struct object_entry). On a large\nrepository like linux-2.6.git (6.5M objects), this saves us 52MB\nmemory.","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/f995629f9e8224a2ea088cc9a343fe76af257c73"},{"sha":"03c9b4ddfb0ae8b2417666a893bb2ea14aa6c215","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: use bitfield for object_entry::dfs_state","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/03c9b4ddfb0ae8b2417666a893bb2ea14aa6c215"},{"sha":"45c3e76c4e8c9935b64c9cdea4262bf7f2864449","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: use bitfield for object_entry::depth\n\nThis does not give us any saving due to padding. But we will be able\nto save once we cut 4 bytes out of this struct in a subsequent patch.\n\nBecause of struct packing from now on we can only handle max depth\n4095 (or even lower when new booleans are added in this struct). This\nshould be ok since long delta chain will cause significant slow down\nanyway.","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/45c3e76c4e8c9935b64c9cdea4262bf7f2864449"},{"sha":"65912e3419d933e8e2724126c0e99faef09a179d","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: note about in_pack_header_size\n\nObject header in a pack is packed really tight (see\npack-format.txt). Even with 8 bytes length, we need 9-10 bytes most,\nplus a hash (20 bytes). Which means this field only needs to store a\nnumber as big as 32 (5 bits).\n\nThis is trickier to pack tight though since a new hash algorithm is\ncoming, the number of bits needed may quickly increase. So leave it\nfor now.","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/65912e3419d933e8e2724126c0e99faef09a179d"},{"sha":"90b42264ec7e22c525aaad27854b42146b7fcf28","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: move in_pack_pos out of struct object_entry\n\nThis field is only need for pack-bitmap, which is an optional\nfeature. Move it to a separate array that is only allocated when\npack-bitmap is used (it's not freed in the same way that objects[] is\nnot). This saves us 8 bytes in struct object_entry.","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/90b42264ec7e22c525aaad27854b42146b7fcf28"},{"sha":"60bb63078b9f7455e5103c4e895a5effa3c55763","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: move in_pack out of struct object_entry\n\nInstead of using 8 bytes (on 64 bit arch) to store a pointer to a\npack. Use an index isntead since the number of packs should be\nrelatively small.\n\nThis limits the number of packs we can handle to 256 (still\nunreasonably high for a repo to work well). If you have more than 256\npacks, you'll need an older version of Git to repack first.\n\nThis technically saves 7 bytes. But we don't see any of that in\npractice due to padding. The saving becomes real when we pack this\nstruct tighter later.","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/60bb63078b9f7455e5103c4e895a5effa3c55763"},{"sha":"70d30f2ee1660db5ac39c9bf503d0d1ae78647cc","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: faster reverse packed_git lookup\n\nWe do a linear search for in_pack index in create_object_entry(). This\nfunction is called for every available object in the worst case (and\non linux-2.6.git, that's about 6.5M). Try to avoid that by saving the\nindex in packed_git. Since we should not have zillions of packs, this\nextra space should not be a big deal.","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/70d30f2ee1660db5ac39c9bf503d0d1ae78647cc"},{"sha":"af22508df05acce81e9639e2c439565c9c77eece","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: refer to delta objects by index instead of pointer\n\nNotice that packing_data::nr_objects is uint32_t, we could only handle\nmaximum 4G objects and can address all of them with an uint32_t. If we\nuse a pointer here, we waste 4 bytes on 64 bit architecture.\n\nConvert these delta pointers to indexes. Since we need to handle NULL\npointers as well, the index is shifted by one [1].\n\nThere are holes in this struct but this patch is already big. Struct\npacking can be done separately. Even with holes, we save 8 bytes per\nobject_entry.\n\n[1] This means we can only index 2^32-2 objects even though nr_objects\n    could contain 2^32-1 objects. It should not be a problem in\n    practice because when we grow objects[], nr_alloc would probably\n    blow up long before nr_objects hits the wall.","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/af22508df05acce81e9639e2c439565c9c77eece"},{"sha":"b9918a38e237f4944e489e4519d725d5a4739192","author":{"name":"Nguyễn Thái Ngọc Duy","email":"d0e3b15a74ff6c70e81434a55b9c843230dc3715@gmail.com"},"message":"pack-objects: reorder 'hash' to pack struct object_entry","distinct":true,"url":"https://api.github.com/repos/pclouds/git/commits/b9918a38e237f4944e489e4519d725d5a4739192"}]}
{"actor":{"display_login":"pclouds"}}