title: Limit HTTP response size on Spring WebFlux
body: <p>I'm writing http crawler using Spring WebFlux and it's easy to parallel execution and to have HTTP timeouts:</p>
  
  <pre><code>val sitesToCrawl: Flux&lt;String&gt; = streamOfUrl()
  val concurrencyLimit = 100
  sitesToCrawl.flatMap(
      { WebClient.create().get().uri(it).exchange().timeout(Duration.ofSeconds(10)) },
      concurrencyLimit
  )
  </code></pre>
  
  <p>But how can I limit page response size, like I don't want to download more than 500KB of data for each URL. Reading HTTP header Content-Length is not reliable. I guess I need to go one level down and use byte buffers and Netty events directly, but would be nice to wrap that using Flux/Mono to keep using those primitives </p>