title: Querying mnesia Fragmentated Tables using QLC returns wrong results
body: <p>am josh in Uganda. i created a mnesia fragmented table (64 fragments), and managed to populate it upto 9948723 records. Each fragment was a disc_copies type, with two replicas.
  Now, using qlc (query list comprehension), was too slow in searching for a record, and was returning inaccurate results.</p>
  
  <p>I found out that this overhead is that qlc uses the select function of mnesia which traverses the entire table in order to match records. i tried something else below.</p>
  
  <pre><code>-define(ACCESS_MOD,mnesia_frag).
  -define(DEFAULT_CONTEXT,transaction).
  -define(NULL,'_').
  -record(address,{tel,zip_code,email}).
  -record(person,{name,sex,age,address = #address{}}).
  
  match()-&gt; Z = fun(Spec) -&gt; mnesia:match_object(Spec) end,Z.
  
  match_object(Pattern)-&gt;
      Match = match(),
      mnesia:activity(?DEFAULT_CONTEXT,Match,[Pattern],?ACCESS_MOD).
  </code></pre>
  
  <p>Trying this functionality gave me good results. But i found that i have to dynamically build patterns for every search that may be made in my stored procedures.</p>
  
  <p>i decided to go through the havoc of doing this, so i wrote functions which will dynamically build wild patterns for my records depending on which parameter is to be searched.</p>
  
  <pre><code>%% This below gives me the default pattern for all searches ::= {person,'_','_','_'}
  
  pattern(Record_name)-&gt;
      N = length(my_record_info(Record_name)) + 1,
      erlang:setelement(1,erlang:make_tuple(N,?NULL),Record_name).
  
  %% this finds the position of the provided value and places it in that 
  %% position while keeping '_' in the other positions.
  %% The caller function can use this function recursively until
  %% it has built the full search pattern of interest
  
  pattern({Field,Value},Pattern_sofar)-&gt;
      N = position(Field,my_record_info(element(1,Pattern_sofar))),
      case N of
          -1 -&gt; Pattern_sofar;
          Int when Int &gt;= 1 -&gt; erlang:setelement(N + 1,Pattern_sofar,Value);
          _ -&gt; Pattern_sofar
      end.
  
  my_record_info(Record_name)-&gt;
      case Record_name of
          staff_dynamic -&gt; record_info(fields,staff_dynamic);
          person -&gt; record_info(fields,person);
          _ -&gt; []
      end.
  
  %% These below,help locate the position of an element in a list
  %% returned by "-record_info(fields,person)"
  
  position(_,[]) -&gt; -1;
  position(Value,List)-&gt;
      find(lists:member(Value,List),Value,List,1).
  
  find(false,_,_,_) -&gt; -1;
  find(true,V,[V|_],N)-&gt; N;
  find(true,V,[_|X],N)-&gt;
      find(V,X,N + 1).
  
  find(V,[V|_],N)-&gt; N;
  find(V,[_|X],N) -&gt; find(V,X,N + 1). 
  </code></pre>
  
  <p>This was working very well though it was computationally intensive.
  It could still work even after changing the record definition since at compile time, it gets the new record info</p>
  
  <p>The problem  is that when i initiate even 25 processes on a 3.0 GHz pentium 4 processor running WinXP, It hangs and takes a long time to return results.</p>
  
  <p>If am to use qlc in these fragments, to get accurate results, i have to specify which fragment to search in like this.</p>
  
  <pre><code>find_person_by_tel(Tel)-&gt;
    select(qlc:q([ X || X &lt;- mnesia:table(Frag), (X#person.address)#address.tel == Tel])).
  
  select(Q)-&gt;
  case ?transact(fun() -&gt; qlc:e(Q) end) of
      {atomic,Val} -&gt; Val;
      {aborted,_} = Error -&gt; report_mnesia_event(Error) 
  end.
  </code></pre>
  
  <p>Qlc was returning [], when i search for something yet when  i use match_object/1 i get accurate results. I found that using match_expressions can help.</p>
  
  <p>mnesia:table(Tab,Props).
  where Props is a data structure that defines the match expression, the chunk size of return values e.t.c</p>
  
  <p>I got a problem when i tried building match expressions dynamically.</p>
  
  <p>Function mnesia:read/1 or mnesia:read/2 requires that you have the primary key</p>
  
  <p>Now am asking myself, how can i efficiently use QLC to search for records in a large fragmented table? Please help.</p>
  
  <p>I know that using tuple representation of records makes code hard to upgrade. This is why
  i hate using mnesia:select/1, mnesia:match_object/1 and i want to stick to QLC. QLC is giving me wrong results in my queries from a mnesia table of 64 fragments even on the same node.</p>
  
  <p>Has anyone ever used QLC to query a fragmented table?, please help</p>