title: How to handle portability issues in a binary file format
body: <p>I'm designing a binary file format to store strings[without terminating null to save space] and binary data.</p>
  
  <p>i. What is the best way to deal with little/big endian systems?
  i.a Would converting everything to network byte order and back with ntohl()/htonl() work?</p>
  
  <p>ii. Will the packed structures be the same size on x86, x64 and arm?</p>
  
  <p>iii. Are their any inherent weakness with this approach?</p>
  
  <pre><code>struct __attribute__((packed)) Header {
      uint8_t magic;
      uint8_t flags;
  };
  
  struct __attribute__((packed)) Record {
      uint64_t length;
      uint32_t crc;
      uint16_t year;
      uint8_t day;
      uint8_t month;
      uint8_t hour;
      uint8_t minute;
      uint8_t second;
      uint8_t type;
  };
  </code></pre>
  
  <p>Tester code I'm using the develop the format:</p>
  
  <pre><code>#include &lt;stdlib.h&gt;
  #include &lt;unistd.h&gt;
  #include &lt;stdio.h&gt;
  #include &lt;limits.h&gt;
  #include &lt;strings.h&gt;
  #include &lt;stdint.h&gt;
  #include &lt;sys/stat.h&gt;
  #include &lt;fcntl.h&gt;
  #include &lt;string.h&gt;
  
  struct __attribute__((packed)) Header {
      uint8_t magic;
      uint8_t flags;
  };
  
  struct __attribute__((packed)) Record {
      uint64_t length;
      uint32_t crc;
      uint16_t year;
      uint8_t day;
      uint8_t month;
      uint8_t hour;
      uint8_t minute;
      uint8_t second;
      uint8_t type;
  };
  
      int main(void)
      {
  
          int fd = open("test.dat", O_RDWR|O_APPEND|O_CREAT, 444);
          struct Header header = {1, 0};
          write(fd, &amp;header, sizeof(header));
          char msg[] = {"BINARY"};
          struct Record record = {strlen(msg), 0, 0, 0, 0, 0, 0, 0};
          write(fd, &amp;record, sizeof(record));
          write(fd, msg, record.length);
          close(fd);
          fd = open("test.dat", O_RDWR|O_APPEND|O_CREAT, 444);
  
  
          read(fd, &amp;header, sizeof(struct Header));
          read(fd, &amp;record, sizeof(struct Record));
          int len = record.length;
          char c;
          while (len != 0) {
              read(fd, &amp;c, 1);
              len--;
              printf("%c", c);
          }
          close(fd);
      }
  </code></pre>