SAP NetWeaver Java on IBM i

• Tuning Guidelines
• Application Recommendations
• Minimum Hardware Recommendations
• Operating System Recommendations
• JVM System Properties

From the hardware to the operating system to the application and workload many factors will influence how a JVM is tuned. Due to the different environments that an SAP NetWeaver system might operate in and the different workloads these systems might run, it is difficult to predict how to optimally configure a JVM without any workload analysis. Through customer experience, benchmarks, and other tests we have learned what levers to pull and knobs to turn in order to tune the JVM. However, the actual settings that are optimal for each environment should be determined by analyzing and making adjustments specifically for that particular environment and workload. Here are some guidelines when tuning a JVM for SAP NetWeaver Java applications.

1. Have reasonable expectations

Performance is a relative term. Most people tend to assess performance based on some sort of comparison. Whether the comparison is valid or invalid it doesn’t matter. It is important to be able to set reasonable expectations for performance and possible improvements based on the current health of your system, resources available, and the workload.

2. Establish performance goals and stick with them

It is important that you consider and decide what your performance goals are. For example, are you tuning for peak load or average load? Are you tuning for optimal flexibility because your workloads vary significantly or are you tuning for optimal performance for one specific workload? Decide what your performance goals are and then tune accordingly.

3. Begin with the recommended settings as a starting point

Your SAP NetWeaver Java system is installed with default settings that provide acceptable performance settings for many workloads. These settings are not meant to be optimal for every workload, but they are meant to be the starting point for tuning the JVM for every workload.

4. Understand tuning and configuration options

It is important for you to understand how to tune your JVM. This includes knowing which settings will have the biggest impact, how different settings are related, and understanding trade-offs.

5. Rules of thumb can help you get started, but are usually not conclusive

A good rule of thumb is not meant to be exact, but rather it is designed to help move you in the right direction. A rule of thumb usually takes into consideration a few of the many variables impacting your system. As such a rule of thumb might move you closer to your target, but there is no substitute for actual workload analysis.

6. Try to measure performance consistently

Measuring performance can be difficult outside of a controlled lab environment. It is important to try and establish some levels of control and consistency if you choose to quantify performance. This includes attempting to make measurements using a workload that is repeatable and fairly consistent.

Back to top

Most SAP NetWeaver Java Application Server configurations are platform independent. However, with certain settings, the operating system, JVM, and the hardware should be considered. The following recommendations are specific to IBM i and the Classic JVM, for other configurations consult SAP documentation.

• Number of Server Nodes
• Number of Threads Per Node

Number of server nodes

Due to the scalability of the IBM i Classic JVM, it is possible to achieve a high level of throughput for each server node. Since each additional server node adds significant memory and processor overhead, limiting the number of server nodes will decrease the memory footprint and processor utilization. A smaller number of server nodes will also reduce the amount of configuration and maintenance.

Recommendation

Add additional server nodes only when the existing, tuned server node(s) does not have sufficient capacity. An indication that an additional server node might be necessary is when performance of the existing server node(s) becomes degraded under an increased load even though the partition where the server node(s) is running has plenty of excess memory and processor capacity. Before adding additional server nodes ensure that the existing nodes have been tuned and are operating efficiently.

Rule of Thumb

One server node per processor
For most workloads it is not necessary to configure more than one server node for each processor dedicated to that workload

Adding Server Nodes

1. Open the SAP J2EE Engine Config Tool
2. Select the instance where the server node will be added
3. From the menu select Server -> Add Server
4. Click "Yes" if prompted to add another server
5. Click "Ok"

Note. If the Native JDBC driver is configured it is necessary to change the driver location of the new server node.

1. Open the SAP J2EE Engine Config Tool
2. Expand the new server node
3. Expand managers
4. Select ConfigurationManager
5. Select rdbms.driverLocation
6. Enter the location of the Native JDBC driver jar file
7. Click "Set"
8. Click "Save"

What Are Server Nodes

Server nodes in the Java Application Server are similar in function to work processes in the ABAP Application Server, but are very different in structure. Both server nodes and work processes are responsible for performing the real work done by the SAP system and both receive that work from the dispatcher.

Whereas a work process is a single threaded job that can only handle one task at a time, a server node consists of a Java Virtual Machine (JVM) that by its nature is multi-threaded and can handle multiple tasks at one time. Because of the multi-threaded nature of the server node, the number of server nodes in a Java Application Server instance will be quite small compared to the number of work processes in an ABAP Application Server instance doing the same amount of work.

Since additional server nodes consume resources, it is important to not add additional server nodes if they are not necessary. The number of server nodes required is dependent on the number of processors available and the expected load. It is important to note that the number of server nodes will not always correspond to the number of processors in a partition.

Number of Threads Per Node

Overview

To maximize the scalability and the capacity of a particular SAP dispatcher or server process, it is possible and sometimes necessary to adjust the number of threads available. Both, server and dispatcher processes have a system threads pool and server processes have an additional application threads pool. For those thread pools you can define an initial number of threads, a minimum value and a maximum value. The system starts with the initial number of threads and adjusts the number of threads available based upon the load between the minimum and maximum value. Ideally, there should always be free threads that are available for application processing.

However, a thread is a resource that has a certain footprint and therefore the number of idling threads should not be excessively high.

To be able to decide if you need to adjust the thread values, you can use the Monitoring application in the Visual Administrator. Open the node that you want to watch, open the services node and click on Monitoring. Then, under root/Kernel you’ll find nodes called "Application Threads Pool" or "System Threads Pool". If you see the "WaitingTasksCount" measure being greater than 0 you might want to increment your MaxThreadCount value. If the "CurrentThreadPoolSize" is constantly higher than InitialThreadPoolSize consider increasing InitialThreadCount.

Following you’ll find the default values and instructions on how to change the Thread Pool values.

Default Values for Thread Pools

Dispatcher System Threads Pool
InitialThreadCount		100
MaxThreadCount		200
MinThreadCount		50

Server System Threads Pool
InitialThreadCount		40
MaxThreadCount		100
MinThreadCount		10

Server Application Threads Pool
InitialThreadCount		40
MaxThreadCount		40
MinThreadCount		40

Changing the Thread Pool configuration

1. Open the SAP J2EE Engine Config Tool
2. Expand Global dispatcher/server configuration, depending on the node that you want to configure
3. Expand managers
4. Select ThreadManager for the System Threads Pool or ApplicationThreadManager for the Application Threads Pool
5. Select InitialThreadCount, MaxThreadCount or MinThreadCount
6. Change the value
7. Click "Set"
8. Click "Save"

Back to top

The minimum hardware requirements are the amount of hardware resources, particularly memory and processors, required to meet the demands of a workload. Note the following about minimum hardware requirements:

• All applications require a certain amount of resources in order to function.
• Running an application in an environment that does not meet minimum resource requirements leads to an increased risk of functional and performance problems if the application runs at all.
• Resource requirements vary from application to application and vary within the same application depending on configuration, workload, etc...

Physical Memory

• Memory requirements include the memory necessary for Java (GC, JIT, JVM), the OS, and the application to start and function properly with acceptable performance.
• Memory usage in the SAP NetWeaver Java environment is coupled tightly with the workload.

• • Memory needs change as the type of workload changes and as the volume of work changes.
• Additional memory beyond minimum requirements is recommended to compensate for peak loads and changes in workload.

The minimum memory requirements will depend on your anticipated workload and usage patterns. For memory recommendations based on your workload a detailed sizing should be completed. Please refer your sizing questions to the IBM SAP International Competence Center (ISICC) at isicc@de.ibm.com.

Processors

• Processor requirements include the processing capacity necessary for Java (GC, JIT, JVM), the OS, and the application to start and function properly with acceptable performance.
• Processor utilization generally increases when sufficient memory is unavailable.
• Additional processing capacity beyond the minimum requirements is recommended to compensate for peak loads and changes in workload.

The minimum processor requirements will depend on your anticipated workload and usage patterns. For processor recommendations based on your workload, a detailed sizing should be completed. Please refer your sizing questions to the IBM SAP International Competence Center (ISICC) at isicc@de.ibm.com.

Back to top

Shared Memory Pools

Shared memory pools provide a method for the operating system to dedicate memory to a specific subsystem or job. Because of the Java Garbage Collection mechanism which touches the whole Java heap once in a while you will want to keep the Java heap in main memory and avoid paging. Therefore, running your Java workload in its own memory pool decreases contention of memory due to other running jobs and reduces the likelihood of parts of the heap being paged out.

An additional benefit for the IBM Classic JVM is that a memory pool provides means to limit the memory resource allocation of a Java system. This is required because with Classic the Java VM parameter -Xmx shall not be used as a heap size limiting parameter. This way you can protect other processes and systems on your IBM i system from excessively growing Java systems.

The disadvantage of configuring a separate memory pool is that the memory is removed from the *BASE pool, which limits how the operating system can manage memory. This could potentially lead to less efficient use of memory resources by the operating system.

Recommendations

• Configure a shared memory pool if your Java instance is mission critical and belongs to a productive system.
• Configure a shared memory pool if your Java instance is consistently memory starved (indicated by excessive paging) due to contention from other workloads on the partition.
• Configure a shared memory pool if it is essential that your Java workload be isolated from other workloads running on your partition.
• Do not configure a shared memory pool if your Java instances are used intermittingly such as in a test or sandbox environment.
• Do not configure a shared memory pool for your Java workload if your system is dedicated to running only that Java workload.

Configuring Shared Memory Pools in an SAP Environment

Follow SAP Note 1023092 "iSeries: Using separate memory pools for Java processes". The size of the shared memory pool should be at least the sum of the actual heap sizes of the server processes, dispatcher and SDM plus some safety buffer of 20%.

Performance adjuster

If you choose to utilize shared memory pools for your Java applications, then consider activating the system Performance Adjuster to allow the system to dynamically manage memory pools.

What Is the Performance Adjuster

The Performance Adjuster allows the IBM i operating system to automatically manage the memory allocated to shared memory pools without any user interaction. This function is activated by default, but is often disabled for SAP workloads.

The Performance Adjuster transfers memory between pools based on the workload behavior. The performance adjuster will automatically adjust the amount of memory allocated to memory pools to compensate for fluctuations in memory utilization, leading to more efficient use of memory resources.

To avoid situations where Java workloads become memory starved after periods of low utilization, the minimum memory pool size should be set to a sufficient value. A maximum memory pools size should also be set to limit the amount of memory any one memory pool can use.

When to Use Performance Adjuster

The Performance Adjuster can be used to better utilize memory resources in certain situations. Consider using the Performance Adjuster if:

• Multiple workloads are running on a single partitions
• Shared memory pools are being utilized
• Memory utilization for a workload fluctuates
• Different workloads have different peak utilization times

How to Change Performance Adjuster

The Performance Adjuster is enabled by setting the QPFRADJ system value. It is recommended to set QPFRADJ to a value of '2' or '3'. This will enable the operating system to periodically check the performance of all the active shared pools and adjust or rearrange the storage and activity levels as needed. If the value is set at ‘2’, then adjustments are also made at IPL time.

Setting Shared Memory Pool Attributes

It is important that Java applications always have the minimum required memory available. Running Java applications with less than the recommended minimum could lead to functional and performance problems. The Performance Adjuster attempts to balance memory between memory pools based on workload behavior, but in certain situations the Performance Adjuster could allocate less memory than the minimum required for a Java application. Therefore it is important to set the appropriate shared memory pool attributes to avoid this situation. If the Performance Adjuster is activated then the following shared memory pool attributes should be set: priority, minimum size, and maximum size. Shared memory pool attributes can be set by using the WRKSHRPOOL command.

Set the priority of the shared memory pool equal to the priority of the base memory pool.

The minimum size and maximum size should be set according to the actual workload behavior. For a rule of thumb and guidelines see SAP Note 49201 “IBM i: Main storage settings for SAP”. For more details see Java memory tuning tips on the IBM i Information Center. Keep in mind that setting values according to your observed workload behavior will determine the optimal settings for your environment.

Max active

The thread activity level of the storage pool containing the SAP NetWeaver Java Application Server instance should be set based on your workload. Increase the max active value as necessary when threads are marked as ineligible (this can be determined using WRKSYSSTS).

Recommendation

If at any time threads are marked as ineligible increase the max active value in increment of 100 until no threads are marked as ineligible.
In most environments and under most workloads it is not necessary to set max active higher than 1000.

How To Change Max Active

1. Enter WRKSHRPOOL command and change the activity level for a specific memory pool under "Max Active" column. This can also be done using the WRKSYSSTS command.

What Is Max Active

The activity level of a storage pool refers to the number of active threads within that storage pool. If the activity level is too low for the number of threads and jobs that are active, your threads are paged out of main storage and marked as ineligible for a short time. This can drastically affect the performance of your application. In prior releases the value of Max Active had an impact on the database optimizer. This is no longer the case.

How to determine if threads are in ineligible state

1. Enter the WRKSYSSTS command
2. Press F11 once to view the Wait->Inel and Active->Inel columns
3. Columns should have value of 0. If the value is greater than 0 the max active value should be increased.

For more details see the Memory pool activity level on the IBM i Information Center (Version 5 Release 4).

Back to top

Most of the tuning for the Java stack is done with JVM system properties. Even though many Java system properties exist and can be modified, most should be left unchanged. For the most part the only JVM system properties that should be maintained are those properties related to the JVM garbage collector; these include maximum heap size (Xmx) and initial heap size (Xms).

For most of the recommendations provided, analysis of the workload, specifically the garbage collector output, will be required in order to determine the appropriate settings.

Recommendations

SAP note 717376 "iSeries: VM Settings for SAP WebAS Java (IBM Classic JVM)" provides the initial settings recommended by SAP. Since it is impossible to provide a single set of values that work optimally in all situations, it is recommended to use these settings as an initial starting point and then make adjustments based on your workload and analysis.

For checking and cleaning up the Java VM parameters you can use the tool CONFIGJVM. If you already made adjustments to the Java heap, you should specify the option KEEPHEAP(*YES) when calling the tool. This preserves any changes made to the Java heap. Details about CONFIGJVM can be found in SAP note 1262494.

Also make sure SAP note 990892 is implemented.

Xmx (maximum heap size)
For the IBM Classic JVM the name of the property, maximum heap size, is somewhat misleading and it is often incorrectly assumed that the Java heap will grow to the value set by Xmx. In a well behaving Java application, the heap should not approach value specified by Xmx. Usually, if the heap size does reach the size indicated by Xmx, this is an indication of an improperly tuned JVM and could potentially lead to degraded performance.

This property if set correctly will not have to be adjusted as workloads change. Maximum heap size should be set to an arbitrarily high value to avoid problems with restricting the heap. With a properly tuned SAP Java Application Server under a full load, you can expect the Java heap for a server node to reach about 4-6 GB in size. Of course, if the server node is not running under a full load, the heap will be less. All other JVMs in the SAP Java Application Server, such as dispatcher and bootstrap JVMs are much smaller than the server JVMs.

The maximum heap size should be set to a value at least 2 GB larger than the largest expected heap size. For the SAP Java Application Server under most conditions, a maximum heap size of 8GB would be sufficient for all JVMs that are part of the Java Application Server, include dispatcher, server nodes, and bootstrap JVMs. Once this value is set correctly it should rarely have to be modified.

Xms (initial heap size)
Since properly setting the value for this parameter is dependent on the workload, it is necessary to analyze the JVM garbage collection statistics when running your workload to determine the optimal value. The optimal value for initial heap size will always be less than maximum heap size and should never be set equal to maximum heap size.

Increase the value of Xms to decrease the frequency of garbage collection cycles. A lower frequency of garbage collection cycles generally leads to a larger heap, but less processor overhead.

Decrease the value of Xms to increase the frequency of garbage collection cycles. A frequency of garbage collection cycles generally leads to a smaller heap, but more processor overhead.

For detailed information on analyzing your SAP NetWeaver Java workload and properly setting the initial heap size property, see Chapter 4 "Configuration and Tuning" in the Redpiece "SAP NetWeaver Java on IBM i5/OS".

Changing System Properties

1. Open the SAP J2EE Engine Config Tool
2. Expand "cluster-data"
3. Expand the instance
4. Click on the component to be changed
5. Modify or add the property in the pane on the right
6. To set the bootstrap JVM properties, click the "Bootstrap" tab

What Are JVM System Properties

JVM system properties change the behavior of an individual JVM. Most JVM system properties are "set and forget" and never need to be changed regardless of workload. Other properties are used to "tune" the JVM and may need to be adjusted occasionally. Properties used for tuning can be impacted by the behavior of the application, workload, and other factors.

Description of Properties

Xmx (maximum heap size)
This property specifies the maximum JVM heap size. The Classic JVM is unique in that the JVM heap should never grow to the size specified by maximum heap size. Though the Xmx value set the limits for heap growth, under normal circumstances it should not influence garbage collector behavior. Xmx is set as a safety precaution in the unlikely event of runaway heap growth. Under normal circumstances the heap should never grow to the size specified with maximum heap size.

Xms (initial heap size)
This property is the primary property used to tune the JVM. The property directly influences the behavior of the garbage collector. The initial heap size value is the threshold used by the garbage collector to know when to initiate a garbage collection cycle. The initial heap size does not refer to the size of the heap at any point during the life of the JVM. The initial heap size value is used to tune the JVM by influencing the frequency of garbage collections.

Back to top

• What's new in 7.1
• Functions by release
• TCP/IP on the 7.1 Information Center

• IBM i InfoCenter
• IBM i Access
• Navigator for i
• HTTP Server