title: How do I lock read/write to MySQL tables so that I can select and then insert without other programs reading/writing to the database?
body: <p>I am running many instances of a webcrawler in parallel.</p>
  
  <p>Each crawler selects a domain from a table, inserts that url and a start time into a log table, and then starts crawling the domain.</p>
  
  <p>Other parallel crawlers check the log table to see what domains are already being crawled before selecting their own domain to crawl.</p>
  
  <p>I need to prevent other crawlers from selecting a domain that has just been selected by another crawler but doesn't have a log entry yet.  My best guess at how to do this is to lock the database from all other read/writes while one crawler selects a domain and inserts a row in the log table (two queries).</p>
  
  <p>How the heck does one do this?  I'm afraid this is terribly complex and relies on many other things.  Please help get me started.</p>
  
  <hr>
  
  <p>This code seems like a good solution (see the error below, however):</p>
  
  <pre><code>INSERT INTO crawlLog (companyId, timeStartCrawling)
  VALUES
  (
      (
          SELECT companies.id FROM companies
          LEFT OUTER JOIN crawlLog
          ON companies.id = crawlLog.companyId
          WHERE crawlLog.companyId IS NULL
          LIMIT 1
      ),
      now()
  )
  </code></pre>
  
  <p>but I keep getting the following mysql error:</p>
  
  <pre><code>You can't specify target table 'crawlLog' for update in FROM clause
  </code></pre>
  
  <p>Is there a way to accomplish the same thing without this problem?  I've tried a couple different ways.  Including this:</p>
  
  <pre><code>INSERT INTO crawlLog (companyId, timeStartCrawling)
  VALUES
  (
      (
          SELECT id
          FROM companies
          WHERE id NOT IN (SELECT companyId FROM crawlLog) LIMIT 1
      ),
      now()
  )
  </code></pre>