title: using parallel processing in google app engine for java
body: <p>I have a series of repetetive jobs where for each job many different websites have to be visited- these range from 100 to 10000 sites per job .</p>
  
  <p>From what I read in Google Documentation for the Task Queue API. a task can be used to send a request to an internal relative URL with some parameters (that are defined as part of the task).</p>
  
  <p>What I want is to be able to control the flow, eg have one 'queue' in which only 50 sites are visited concurrently as part of one job, then one faster queue where for one job as many as 5000 sites are visited concurrently...</p>
  
  <p>How do I accomplish the above in Google App Engine for Java?</p>
  
  <p>The only solution that I could think of is a parallel processing framework like Korus, but that does not provide me with the level of control that is provided by Task Queues...Is there some easy and/or better way of accomplishing what I want?</p>