Take the 2-minute tour ×

I'm writing a Scala library to make querying an paginated JSON API easier. Each API call returns an object that looks something like this:

  "count": 100,
  "current_page": 1,
  "total_pages": 2,
  "records": [

I'd like to have a function that returned some sort of iterator like MyIterator[Record]. Are there any standard ways to do this in the Scala world, perhaps even constructs in the standard library that could help me?

I'm using lift-json generally for my JSON parsing, if that's helpful.


share|improve this question
add comment

1 Answer

up vote 4 down vote accepted

If you have an Iterator[Iterator[A]] you can use flatten to produce an Iterator[A] which chains all the nested Iterators together:

scala> Iterator(Iterator(1, 2), Iterator(3, 4)).flatten.toList

Combined with one of the factory methods on the Iterator companion object, you could probably turn this into a one-liner:

Iterator.range(1, pageCount).map(i: Int => fetchPage(i)).flatten

It might need to be a bit more complex if you can only get the page count by retrieving a page... something like:

val (firstPage, pageCount) = getFirstPageWithCount()
firstPage ++ (Iterator.range(2, pageCount).map(fetchPage).flatten)
share|improve this answer
Thanks, David, that's very helpful. The API I'm targeting does only give the page count with the first page, so I'll have to use the second approach. Is there any way to only fetch subsequent pages if necessary? That is, a sort of lazy initialization? –  pr1001 Jan 29 '11 at 23:06
Actually, the example I gave you is lazy already. (Try Iterator.range(1, 10).map { x => println(x); x } .take(3).toList to verify this yourself.) –  David Winslow Jan 29 '11 at 23:14
Touché! Very cool, thanks! By the way, when might a Stream be more appropriate? –  pr1001 Jan 29 '11 at 23:31
stackoverflow.com/questions/1527962/… discusses the differences quite nicely, I think. –  David Winslow Jan 29 '11 at 23:43
add comment

Your Answer


