title: JasperReport GC overhead with huge report
body: <p>I am using jasper report to generate a huge Excel report that might extend to 100k+ lines. </p>
  
  <p>I first get my report lines then feed the jasperReport object with them. This is done without a problem. </p>
  
  <p>The fillReport method takes some time but seems to be doing okay as well. </p>
  
  <p>But when exporting my report I get huge memory usage and eventually my GC gets overloaded and my app crashes. For the record, this is the exception I'm getting:</p>
  
  <pre><code>java.lang.OutOfMemoryError: GC overhead limit exceeded
      at java.util.Arrays.copyOf(Unknown Source)
      at java.io.ByteArrayOutputStream.grow(Unknown Source)
      at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)
      at java.io.ByteArrayOutputStream.write(Unknown Source)
      at java.io.ObjectOutputStream$BlockDataOutputStream.write(Unknown Source)
      at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
      at java.io.ObjectOutputStream.defaultWriteObject(Unknown Source)
      at net.sf.jasperreports.engine.fill.JRTemplatePrintElement.writeObject(JRTemplatePrintElement.java:363)
      at sun.reflect.GeneratedMethodAccessor359.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
      at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
      at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
      at java.io.ObjectOutputStream.writeObject0(Unknown Source)
      at java.io.ObjectOutputStream.writeObject(Unknown Source)
      at java.util.ArrayList.writeObject(Unknown Source)
      at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
      at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
      at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
      at java.io.ObjectOutputStream.writeObject0(Unknown Source)
      at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
      at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
      at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
      at java.io.ObjectOutputStream.writeObject0(Unknown Source)
      at java.io.ObjectOutputStream.writeObject(Unknown Source)
      at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.writeData(JRAbstractLRUVirtualizer.java:704)
      at net.sf.jasperreports.engine.fill.JRSwapFileVirtualizer.pageOut(JRSwapFileVirtualizer.java:87)
      at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.virtualizeData(JRAbstractLRUVirtualizer.java:664)
  </code></pre>
  
  <p>I would like to know whether there's a way to optimize this use of memory and prevent this crash, specially when I need to be able to let my users generate these reports simultaneously. I have tried using swap files and virtualizers but this seems to be only optimizing the fillreport process.</p>
  
  <p>This is the bit of code where I call my report generation function:</p>
  
  <pre><code> List&lt;LineaInformeFacturacionExcelCVO&gt; listaLineasInformeFacturacionExcelCVO = new ArrayList&lt;LineaInformeFacturacionExcelCVO&gt;();
          List&lt;LineaInformeFacturacionExcel&gt; lineasInformeFacturacionExcel = lineaInformeFacturacionExcelDao.getLineasInformeFacturacionExcel(filtros);
  
  getBytesInformeExcel(reportRealPath, parameters, listaLineasInformeFacturacionExcelCVO, true, jrExporterParameters);
  </code></pre>
  
  <p>This is the function itself:</p>
  
  <pre><code>    public byte[] getBytesInformeExcel(String reportRealPath, Map&lt;String, Object&gt; parameters, List&lt;?&gt; list, boolean virtualize, Map&lt;JRExporterParameter, Object&gt; jrExporterParameters) throws EmptyReportException, JRException {
  
  
      if (list != null &amp;&amp; list.isEmpty()) {
                  throw new EmptyReportException();
              }
  
                // Acordarse de pasarlo entre subrepors
              if (parameters == null)
                  parameters = new HashMap&lt;String, Object&gt;();
  
              Locale spainLocale = new Locale("es","ES");     
              TimeZone europeMadridTimezone = TimeZone.getTimeZone("Europe/Madrid");                  
              parameters.put(JRParameter.REPORT_LOCALE, spainLocale);
              parameters.put(JRParameter.REPORT_TIME_ZONE, europeMadridTimezone);     
  
              JRSwapFileVirtualizer virtualizer = null;
              if (virtualize) {           
                  virtualizer = new JRSwapFileVirtualizer(250, new JRSwapFile(System.getProperty("java.io.tmpdir"), 250, 250));
                  parameters.put(JRParameter.REPORT_VIRTUALIZER, virtualizer);                    
              }
  
              parameters.put(JRParameter.IS_IGNORE_PAGINATION, false);
  
              JasperReport jasperReport;
  
              byte[] reportBytes = null;
              JasperPrint jasperPrint = null;
  
              try {
  
                  jasperReport = (JasperReport) JRLoader.loadObjectFromFile(reportRealPath);          
  
                  Crono crono = new Crono();
                  crono.startCrono();
                  LOG.info("Iniciando JasperFillManager.fillReport() ");
  
                  if (list == null) {
                      Connection connection = lineaInformeFacturacionDao.obtenerConnection();          
                      jasperPrint = JasperFillManager.fillReport(jasperReport, parameters, connection);
                  }
                  else {
                      jasperPrint = JasperFillManager.fillReport(jasperReport, parameters, new JRBeanCollectionDataSource(list));
                  }
  
                  LOG.info("Fin JasperFillManager.fillReport() :" + crono.stopCrono() + " ms.");
  
                  // Comprueba si el report está vacío.
                  if (jasperPrint.getPages().isEmpty() || jasperPrint.getPages().get(0).getElements().isEmpty()) {
                         throw new EmptyReportException();
                  }       
  
                  ByteArrayOutputStream xlsReport = new ByteArrayOutputStream();
  
                  JRExporter exporter = new JRXlsExporter();;
                  String formato = parametrosConfiguracion.getPropertie("informes.excel.formato");
                  if(StringUtils.isNotBlank(formato)) {
                      if(formato.equals(parametrosConfiguracion.getPropertie("excel.formato.xls"))) {
                          exporter = new JRXlsExporter();
                      } else if(formato.equals(parametrosConfiguracion.getPropertie("excel.formato.xlsx"))) {
  
                          exporter = new JRXlsxExporter();
                      }
                  }                        
  
                  exporter.setParameter(JRXlsExporterParameter.JASPER_PRINT, jasperPrint);
                  exporter.setParameter(JRXlsExporterParameter.OUTPUT_STREAM, xlsReport);
  
                  // Carga los parámetros por defecto
                  exporter.setParameter(JRXlsExporterParameter.IS_ONE_PAGE_PER_SHEET, Boolean.FALSE);
                  exporter.setParameter(JRXlsExporterParameter.IS_DETECT_CELL_TYPE, Boolean.TRUE);
                  exporter.setParameter(JRXlsExporterParameter.IS_WHITE_PAGE_BACKGROUND, Boolean.FALSE);
                  exporter.setParameter(JRXlsExporterParameter.IS_REMOVE_EMPTY_SPACE_BETWEEN_ROWS, Boolean.TRUE);
  
                  // Carga estos parámetros pasados como parámetro sobreescribiendo los por defecto 
                  if (jrExporterParameters != null) {
                      exporter.setParameters(jrExporterParameters);
                      exporter.setParameter(JRXlsExporterParameter.JASPER_PRINT, jasperPrint);
                      exporter.setParameter(JRXlsExporterParameter.OUTPUT_STREAM, xlsReport);
                  }            
  
                  crono = new Crono();
                  crono.startCrono();
                  LOG.info("Iniciando exporter.exportReport() ");  
                  exporter.exportReport();
                  LOG.info("Fin exporter.exportReport() :" + crono.stopCrono() + " ms.");
  
                  reportBytes = xlsReport.toByteArray();     
  
  
              }finally{
                  if (virtualize) {
                      virtualizer.cleanup();
                  }
              }
  
              return reportBytes;
  }
  </code></pre>
  
  <p>And finally, if this is of any help, here's a screenshot of a memory profiling program at the moment of generating this report:</p>
  
  <p><img src="https://i.stack.imgur.com/YfqNI.png" alt="enter image description here"></p>
  
  <p>Crash happens at the second GC use peak.</p>
  
  <p>Exporting directly to File with </p>
  
  <pre><code>exporter.setParameter(JRXlsExporterParameter.OUTPUT_FILE, new File("c://archivo.xlsx"));
  </code></pre>
  
  <p>instead of a ByteStream produces the following exception:</p>
  
  <pre><code>java.lang.OutOfMemoryError: Java heap space
      at java.util.Arrays.copyOf(Unknown Source)
      at java.io.ByteArrayOutputStream.grow(Unknown Source)
      at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)
      at java.io.ByteArrayOutputStream.write(Unknown Source)
      at java.io.ObjectOutputStream$BlockDataOutputStream.drain(Unknown Source)
      at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(Unknown Source)
      at java.io.ObjectOutputStream.writeObject0(Unknown Source)
      at java.io.ObjectOutputStream.writeObject(Unknown Source)
      at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.writeData(JRAbstractLRUVirtualizer.java:704)
      at net.sf.jasperreports.engine.fill.JRSwapFileVirtualizer.pageOut(JRSwapFileVirtualizer.java:87)
      at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.virtualizeData(JRAbstractLRUVirtualizer.java:664)
      at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.evict(JRAbstractLRUVirtualizer.java:485)
      at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.requestData(JRAbstractLRUVirtualizer.java:630)
      at net.sf.jasperreports.engine.base.ElementsBlock.ensureData(VirtualizableElementList.java:463)
      at net.sf.jasperreports.engine.base.ElementsBlock.ensureDataAndTouch(VirtualizableElementList.java:432)
      at net.sf.jasperreports.engine.base.ElementsBlock.get(VirtualizableElementList.java:283)
      at net.sf.jasperreports.engine.base.ElementsBlockList.get(VirtualizableElementList.java:717)
      at net.sf.jasperreports.engine.base.VirtualizableElementList.get(VirtualizableElementList.java:96)
      at net.sf.jasperreports.engine.base.VirtualizableElementList.get(VirtualizableElementList.java:54)
      at net.sf.jasperreports.engine.export.JRGridLayout.createWrappers(JRGridLayout.java:922)
      at net.sf.jasperreports.engine.export.JRGridLayout.&lt;init&gt;(JRGridLayout.java:140)
      at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportPage(JRXlsAbstractExporter.java:1000)
      at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReportToStream(JRXlsAbstractExporter.java:983)
      at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReport(JRXlsAbstractExporter.java:650)
      at com.services.informes.InformesService.getBytesInformeExcel(InformesService.java:920)
      at com.services.informes.InformesService.getBytesInformeFacturacionExcel(InformesService.java:277)
      at com.controllers.InformesController.exportarInformeFacturacionExcel(InformesController.java:391)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:212)
  </code></pre>
  
  <p>This is the memory profiling in this case:
  <img src="https://i.stack.imgur.com/UGCs3.png" alt="enter image description here"></p>