title: Optimization in obtaining a list of IP conflicts from a Dataset
body: <p>I have a list of Network data containing the ID No., it's network IP and virtual IP. Sample shown below.</p>
  
  <pre><code>    ID  Network IP    Virtual IP
      1   10.1.1.0/24   -
      2   10.2.2.0/24   -
      3   10.3.3.0/24   10.4.4.88
      4   10.4.4.0/24   -
      5   10.1.0.0/16   -
      6   ...
      ...
      ...
  </code></pre>
  
  <p>The objective is to detect and output if there's any Network IP or Virtual IP conflicts/overlaps within the list which includes the resulting Network IP that conflicts and a Remark. <code>eg. Network IP conflict --&gt; 10.1.1.0/24 overlaps with 10.1.0.0/16</code> and <code>Virtual-ip conflict --&gt; 10.4.4.88 is in range of 10.4.4.0/24 (of different ID)</code></p>
  
  <p>Here's a sample output of the code.</p>
  
  <pre><code>ID  Network IP    Virtual IP    Result        Remark
  5   10.1.0.0/16   -             10.1.0.0/16   Network Conflict
  1   10.1.1.0/24   -             10.1.0.0/16   Network Conflict
  3   10.3.3.0/24   10.4.4.88     10.4.4.0/24   Virtual-ip Conflict
  4   10.4.4.0/24   -             10.4.4.0/24   Virtual-ip Conflict
  </code></pre>
  
  <p>Below is the sample code and currently I'm using nested "for" loops which is pretty slow. Hence, am just wondering if there's a more efficient algorithm to resolve this especially when it comes to comparing 100k+ Data?</p>
  
  <pre><code>import ipaddress
  
  class Network:
      def __init__(self, ID='-', IPNet='-', VIP='-', Result='-', Remark='-'):
          self.ID = ID          # ID Number
          self.IPNet = IPNet    # Network IP
          self.VIP = VIP        # Virtual IP
          self.Result = Result  # Resulting Network IP that conflicts
          self.Remark = Remark
      def __eq__(self, other):
          return self.ID == other.ID and self.IPNet == other.IPNet and self.VIP == other.VIP and \
                 self.Result == other.Result and self.Remark == other.Remark
      def __hash__(self):
          return hash((self.ID, self.IPNet, self.VIP, self.Result, self.Remark))
  
  Final_List = []
  Input_List = [Network('1', '10.1.1.0/24', '-', '-', '-'),
                Network('2', '10.2.2.0/24', '-', '-', '-'),
                Network('3', '10.3.3.0/24', '10.4.4.88', '-', '-'),
                Network('4', '10.4.4.0/24', '-', '-', '-'),
                Network('5', '10.1.0.0/16', '-', '-', '-')]
  
  for in1 in Input_List:
      IPNet1 = in1.IPNet
      for in2 in Input_List:
          IPNet2 = in2.IPNet
  
          # Resolving overlapping Network IP
          if in1.ID != in2.ID and ipaddress.ip_network(IPNet1).overlaps(ipaddress.ip_network(IPNet2)):
              Remark = 'Network Conflict'
              Result = '-'
  
              # Comparing size of network range. Resulting network IP will be the one with larger size
              size1 = ipaddress.ip_network(IPNet1).num_addresses
              size2 = ipaddress.ip_network(IPNet2).num_addresses
              if size1 &gt;= size2:
                  Result = IPNet1
              else:
                  Result = IPNet2
  
              _in1 = Data(in1.ID, IPNet1, in1.VIP, Result, Remark)
              Final_List.append(_in1)
              _in2 = Data(in2.ID, IPNet2, in2.VIP, Result, Remark)
              Final_List.append(_in2)
              break
          # Resolving Virtual IP conflicts
          elif in1.ID != in2.ID and in1.VIP != '-' and ipaddress.ip_address(in1.VIP) in ipaddress.ip_network(IPNet2):
              Remark = 'Virtual-ip Conflict'
              Result = IPNet2
              _in1 = Data(in1.ID, IPNet1, in1.VIP, Result, Remark)
              Final_List.append(_in1)
              _in2 = Data(in2.ID, IPNet2, in2.VIP, Result, Remark)
              Final_List.append(_in2)
              break
  
  Final_List = list(set(Final_List))  # Remove duplicates
  Final_List = sorted(Final_List, key=lambda x: (ipaddress.ip_network(x.IPNet), x.ID), reverse=False)
  </code></pre>