Ok, I'm pretty sure that the problem is with serializing and deserializing the result, which is done on the main thread.
When I changed the kotlin code to only return the first 5 entries (after fetching all of them), there were no lost frames.