# IntroFor this challenge a "large file" exists in /home/big on a remote server. The challenge states that there is a present within the large file. Based on this information, we need to determine what important bytes are in this file hoping that it's the flag. Tools such as dd are also restricted on the remote host.
## Initial ResearchAfter SSH'ing into the box we examine the size of the file:```bashbig@ee6e73167aa0:~$ ls -lah my_huge_file -rw-r--r-- 1 root root 16T May 10 08:33 my_huge_file```We try to examine the contents of the file:```bashbig@ee6e73167aa0:~$ cat my_huge_file ?^C```We notice that we can scp files to a tmp directory that we create for the team:```bashscp -P 9000 dd [email protected]:/tmp/libwtf/```Now we can run dd locally to carve out a 4096 bytes of the file:```bash./dd if=/home/big/my_huge_file skip=1 bs=1 count=4096 >>1blk```Since some hex editors were restricted we decided to use od instead. The od command in Linux is used to convert the content of input in different formats with octal format as the default format. We can see the existence of a lot of zeros:```bashbig@ee6e73167aa0:/tmp/libwtf$ od -b 1blk 0000000 237 230 217 000 000 000 000 000 000 000 000 000 000 000 000 0000000020 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000*0010000```
## More researchAfter doing some googling we found this https://wiki.archlinux.org/index.php/sparse_file. A sparse file is a type of computer file that attempts to use file system space more efficiently when blocks allocated to the file are mostly empty. This is achieved by writing brief information (metadata) representing the empty blocks to disk instead of the actual "empty" space which makes up the block, using less disk space. The full block size is written to disk as the actual size only when the block contains "real" (non-empty) data.
Knowing this we can get the real size of the data within the file using linux tools:```bashbig@ee6e73167aa0:~$ du -h my_huge_file 32K	my_huge_file```
We now know the data within the file is very small and we need to carve it out. But, how can we do this effectively? Luickly we have the following option params in the lseek function to speed this up (http://man7.org/linux/man-pages/man2/lseek.2.html).```bashSEEK_DATA - Adjust the file offset to the next location in the file greater than or equal to offset containing data. If offset points to data, then the file offset is set to offset.
SEEK_HOLE - Adjust the file offset to the next hole in the file greater than or equal to offset.  If offset pointsinto the middle of a hole, then the file offset is set to offset.  If there is no hole past offset, then the fileoffset is adjusted to the end of the file (i.e., there is an implicit hole at the end of any file).```
## The SolutionLuckily we found a C program online to find the offsets of non-zero data within the large file instead of writing my own. Shoutout to xkikeg https://gist.github.com/xkikeg/4645373.
After copying and compiling the C program to find data positions of sparse file with SEEK_DATA & SEEK_HOLE we can run it on the remote system. The following values are returned:```bash0x0 0x10000x47868c00000 0x47868c010000x77359400000 0x773594010000xa6e49c00000 0xa6e49c010000xe57a5680000 0xe57a56810000xffffe380000 0xffffe3810000xfffff708000 0xfffff708042 - 17,592,176,640,000```
Now that we have the offsets of where data exists we can use our local version of dd to carve out the data. The skip value is the start of each section and the count values is the start position of the data subtracted by the end position. We also converted the hex data to decimal. 
Example for computing the offset of last data section within the file - 0xfffff708000 = 17,592,176,640,000:```bashbig@ee6e73167aa0: ./dd if=/home/big/my_huge_file skip=17592176640000 bs=1 count=4096 >>outfin```
When we do this for all blocks we get the flag:```bashbig@ee6e73167aa0:/tmp/libwtf$ cat outfin ?shkCTF{sp4rs3_f1l3s_4r3_c001_6cf61f47f6273dfa225ee3366eacb8eb}```