title: Parsing large log file - Python
body: <p>I have a firewall log file which looks like this:</p>
  <blockquote>
  <p>&quot;No.&quot;,&quot;Time&quot;,&quot;Source&quot;,&quot;Destination&quot;,&quot;Protocol&quot;,&quot;Info&quot;
  &quot;1&quot;,&quot;0.000000&quot;,&quot;172.16.113.168&quot;,&quot;172.16.112.50&quot;,&quot;TELNET&quot;,&quot;Telnet Data
  ...&quot;  &quot;2&quot;,&quot;0.000426&quot;,&quot;172.16.112.50&quot;,&quot;172.16.113.168&quot;,&quot;TELNET&quot;,&quot;Telnet
  Data ...&quot;
  &quot;3&quot;,&quot;0.019849&quot;,&quot;172.16.113.168&quot;,&quot;172.16.112.50&quot;,&quot;TCP&quot;,&quot;21582 &gt; telnet
  [ACK] Seq=2Ack=2   Win=32120 Len=0&quot;
  &quot;4&quot;,&quot;0.530125&quot;,&quot;172.16.113.168&quot;,&quot;172.16.112.50&quot;,&quot;TELNET&quot;,&quot;Telnet Data
  ...&quot;  &quot;5&quot;,&quot;0.530634&quot;,&quot;172.16.112.50&quot;,&quot;172.16.113.168&quot;,&quot;TELNET&quot;,&quot;Telnet
  Data ...&quot; &quot;6&quot;,&quot;0.549962&quot;,&quot;172.16.113.168&quot;,&quot;172.16.112.50&quot;,&quot;TCP&quot;,&quot;21582</p>
  <blockquote>
  <p>telnet [ACK] Seq=3 Ack=3 Win=32120 Len=0&quot;</p>
  </blockquote>
  </blockquote>
  <p>I want to be able to run the file by its name (I am using Linux) eg.</p>
  <p><strong>log1.py logfile.csv</strong> (name of the program followed by the name of the logfile)
  and get the following output:</p>
  <blockquote>
  <p><strong>$ log1.py logfile.csv</strong>
  Source IP      Destination IP   Protocol      Count</p>
  <pre><code>        0.0.0.0     255.255.255.255      BOOTP         20
      0.1.125.174         131.84.1.31        TCP          2
      192.168.1.1         172.168.1.2        TCP        100 
  
         (............lots more here .....................)
  
  Oracle_89:a5:9f       3com_9c:b2:54        ARP         14
  
                                          Total:     649787
  </code></pre>
  </blockquote>
  <p>And another very useful feature I would like to have is when i run the program with a source IP address and destination IP address. I would like the output to look something similar to the following:</p>
  <blockquote>
  <p><strong>$ log1.py 172.16.112.50 logfile.csv</strong></p>
  <pre><code>      Source IP      Destination IP   Protocol      Count
  
    172.16.112.50      135.13.216.191        IMF          4
                                            SMTP         53
                                             TCP         43
                                          TELNET         35
          (............lots more here .....................)
  
                       172.16.112.194       SMTP          7
                                             TCP         42
                                          TELNET       3745
  
                                          Total:      38369
  </code></pre>
  </blockquote>
  <p>And finally, I would like to be able to specify both the source IP address and destination IP and adress and get the following output:</p>
  <blockquote>
  <p><strong>$ log1.py 172.16.112.50 202.77.162.213 packets.csv</strong>
  Source IP      Destination IP   Protocol      Count</p>
  <pre><code>  172.16.112.50      202.77.162.213       ICMP          1
                                         Portmap          5
                                             RSH          9
                                         SADMIND          1
                                             TCP         30
                                          TELNET         41
  
                                          Total:         87
  </code></pre>
  </blockquote>
  <p>I am a junior systems administrator and don't really have a lot of experience with programming (just HTML) I have started learning however, I have been stuck on this problem for the past 3 days here is what I have so far:</p>
  <pre><code># Function for validating IP address is valid or not 
  def ip_validation(ip_address):
      ip_regex= re.match('^[\d]{1,3}[.][\d]{1,3}[.][\d]{1,3}[.][\d]{1,3}$', ip_address)
      return ip_regex
  def filereader(file_name):
      file_dump= open(file_name,'r')
      for eachline in file_dump:
          line_a= eachline.replace('\&quot;','') # removes all quotes from the file
          line_b= line_a.split(',') # Delimate each fild based on ','             
          src_ip= line_b[2] # Source IP
          dst_ip= line_b[3] # Destination IP
          prot= line_b[4] # Protocol
          eachline= src_ip, dst_ip, prot      
          itlist.append(eachline) 
          itlist.sort()
          print itlist
  </code></pre>