Skip to end of metadata
Go to start of metadata

This section describes some recommended performance tuning configurations to optimize BAM. It assumes that you have set up the BAM server on Unix/Linux, which is recommended for a production deployment. If you have high volumes of data with high concurrency, it is recommended to use a distributed BAM setup. For instructions, see product deployment and clustering guide.


OS-Level Settings


    net.ipv4.tcp_fin_timeout = 30
    fs.file-max = 2097152
    net.ipv4.tcp_tw_recycle = 1
    net.ipv4.tcp_tw_reuse = 1
    net.core.rmem_default = 524288
    net.core.wmem_default = 524288
    net.core.rmem_max = 67108864
    net.core.wmem_max = 67108864
    net.ipv4.tcp_rmem = 4096 87380 16777216
    net.ipv4.tcp_wmem = 4096 65536 16777216
    net.ipv4.ip_local_port_range = 1024 65535      



    * soft nofile 4096
    * hard nofile 65535

    Optimal values for these parameters depend on the environment.


    * soft nproc 20000
    * hard nproc 20000

JVM settings


To avoid this issue, you can define a maximum level of entity substitutions that the XML parser allows in the system. You do this using the entity expansion limit attribute that is in the <BAM_HOME>/bin/wso2server.bat file (for Windows) or the <BAM_HOME>/bin/wso2server.sh file (for Linux/Solaris). The default entity expansion limit is 64000.



WSO2 Carbon platform-level settings



JDBC Pool Configurations

Within the WSO2 platform, we use Tomcat JDBC pooling as the default pooling framework due to its production ready stability and high performance. The table below indicates some recommendations on how to configure the JDBC pool using the <PRODUCT_HOME>/repository/conf/datasources/master-datasources.xml file. For more details about recommended JDBC configurations, see The Tomcat JDBC Connection Pool.



This value should match the maximum number of requests that can be expected at a time in your production environment. This is to ensure that, whenever there is a sudden increase in the number of requests to the server, all of them can be connected successfully without causing any delays. Note that this value should not exceed the maximum number of requests allowed for your database.

The indication of whether connection objects will be validated before they are borrowed from the pool. If the object validation fails, it will be dropped from the pool, and we will attempt to borrow another connection.

Setting this property to 'true' is recommended as it will avoid connection requests from failing. The validationQuery property should be used if testOnBorrow is set to true. To increase the efficiency of connection validation and to improve performance, validationInterval property should also be used.


To avoid excess validation, run validation at most at this frequency (time in milliseconds). If a connection is due for validation, but has been validated previously within this interval, it will not be validated again. The default value is  30000  (30 seconds).

This time out can be as high as the time it takes for your DBMS to declare a connection as stale. For example, MySQL will keep a connection open for as long as 8 hours, which requires the validation interval to be within that range. However, note that having a low value for validation interval will not incur a big performance penalty, specially when database requests have a high throughput. For example, a single extra validation query run every 30 seconds is usually negligible.


WSO2 BAM specific settings

The values discussed below are general recommendations. They might not be optimal for the specific hardware configurations in your environment. We recommend you to carry out load tests on your environment to tune BAM accordingly.

Improvement AreaPerformance Recommendations
Data receiver nodes
  • Xms1024m -Xmx1024m -XX:MaxPermSize=512m
  • Change the /etc/security/limits.conf with the following values:

    * soft nofile 4096
    * hard nofile 65535

    You need not change these nofile values, if you already done it in OS-Level Settings.

Analyzer nodesXms1024m -Xmx1024m -XX:MaxPermSize=512m
Dashboard nodesXms1024m -Xmx1024m -XX:MaxPermSize=512m
Hadoop nodes
  • Recommended OS: Linux
  • At least 10 GB storage capacity in each node
  • At least 100 Mbps network bandwidth
  • Set hadoop.root.logger=ERROR
Cassandra nodes
  • Have separate disks for your commit log and data dirs (sstables)
  • Set the Heap memory as follows:

    System memoryHeap size
    Less than 2 GB1/2 of system memory
    2-4 GB1 GB
    More than 4 GB1/4 system memory, but not more than 8GB
  • Set following configuration in cassandra.yaml according to your hardware resources:

    • concurrent_reads: 4 * no of cores
    • concurrent_writes: 8 * no of CPU cores

For more information see Apache Cassandra 1.0 Documentation.

Memory size of Hive

In order to increase the memory size of Hive try the following configurations:

  • Increase the setting for the Hadoop heapsize. For example increase it to 4096 as shown below:
    export HADOOP_HEAPSIZE=4096
  • Add the below entry in the <PRODUCT_HOME>/repository/conf/advanced/hive-site.xml file:  
   <value> -server -Xmx512M</value>  

  • No labels