This page lists information about the problems that have been encountered with persistent connections. It also will give you some things to consider in setting your persistent connection configurations in your IBM HTTP Server.
Contents
Prior to persistent connections, the browser would open a connection for each HTTP request and the web server would send the response and close the connection. This presents some obvious questions as to the performance of the network as well as the client-to-server communication.
So, HTTP 1.1 specifications created persistent connections. This allows the browser to send multiple HTTP requests on a single connection. This capability is negotiated in the response to the first request on a connection. The server can choose how many requests it will allow on a persistent connection and also how long to wait for subsequent requests before terminating the connection. Most servers will allow you to configure these things.
Then we started to see more problems but only with SSL connections. This is because SSL connections are a "tunnel" from the browser to the server. Normal requests generally come through an Internet Service Provider server and other proxy servers, so the persistent connections to the content server is with the proxy or ISP, not with the browser.
Over the past few months, we have encountered many problems when SSL was being used. These problems included the following symptoms:
- Netscape 4.0 and 4.5 hanging
- Internet Explorer 4.0 and 5.0 displaying a window indicating the server sent it incorrect data
- Partial documents and images on both Netscape and Internet Explorer
- Browser's switching a connection from SSL to normal
- Other unexplained strange occurrences
We spent considerable time researching this and determined that the problem lies in the browser and the client TCP/IP stack.
The only way to explain this is to get down to the technical level. Imagine the browser reading and parsing the HTML document. It finds an image, sees that persistent connections are supported, and sends the request for the image on the existing connection to the server (just like it should). If there is quite a bit of data that is being sent for the HTML document, it will be broken down into multiple TCP packets and pacing will need to be done. If the persistent timeout is relatively small, then the server could time out the persistent connection before the browser has received the entire HTML document. The server was closing the connection. The method chosen to close the connection causes a TCP FIN to be sent on the connection and subsequently causes the TCP layer to send a RESET response to any future data that is received on that connection. The RESET was causing the browser to discard whatever part of the HTML document it had not received. One would expect the browser to simply re-run the request, but it does not do this.
Our only circumvention up to now has been to turn persistent connections off in the server. You do this by setting MaxPersistRequest to a value of 1.
We have spent some time trying to determine how we could work around this problem and finally came up with a workaround "fix". It is important to remember that this is a work-around for a timing problem on the browser. It has been proven to work in most instances, but it is still possible to hit this timing window. The work-around just moves the timing window and narrows it as much as possible. This "fix" is in V4R3 PTF SF59666 and V4R4 PTF SF59667. It will be in the base for the next release.
In this workaround, the server shuts down the connection to the browser in such a way that the TCP stack on the iSeries will not reject the inbound data with a RESET. The server will now cause TCP to send the TCP FIN for the connection to the client, then wait a configurable amount of time for the client to close the connection. During this time, the server ignores any inbound requests (which it should do, the browser will send new requests). This time defaults to 60 seconds and as I mentioned is configurable using a new HTTP server directive "CloseTimeout". You will have to use the WRKHTTPCFG green screen editor to add or change this directive. Its only parameter is a time value similar to what you use for PersistTimeout. We have seen that under heavy load situations when the server is sending back larger HTML documents, that this problem can still occur with the default 60 second timeout. So, you may have to adjust the CloseTimeout directive to an even higher value to get around the client browser problem.
Well, first, you need to decide if persistent connections is something you want. You should know that the server was shipped to use persistent connections by default (this was changed in PTF SF60957 for V4R3 and PTF SF60604 for V4R4 so the default is now that persistent connections are disabled) , so if it is not something you want, you may need to make a change to your web server configuration.
Persistent connections is really a double-edged sword. On one hand, when your site uses SSL, your site visitors/customers can use one SSL connection to process multiple HTTP requests therefore avoiding the "SSL handshake" for each request. Using Persistent connections will improve the performance and help the network out because fewer SSL connections will be needed to serve your HTML documents.
On the other hand, when a persistent connection is used, the "thread" of the server that the persistent connection is using is dedicated to that browser. The browser does not close the connection when it gets its last response from the server. It assumes that the user will likely select a link from that page and go to the same server with another request. So it will not close the connection unless the next link or request is for a different server. And if the user does not do anything, the browser will still not close the connection. So, the "thread" not only is tied to that browser for "think" time, it also could be tied to the browser if they walk away from their computer or power it off without closing their browser. This is where the PersistTimeout and new CloseTimeout comes into play. You can control the amount of time that the server allows for "think" time on the persistent connection. While the server "thread" is waiting, it cannot process any other requests. So, it is possible that new visitors/customers on your site will get their requests queued up waiting for the persistent connections to time out. So, you now may be impacting the performance of new visitors/customers on your site.
So, should you allow persistent connections or not. The answer is really something that you will have to decide for your own web site. It really depends on how much traffic your site has, what kind of traffic it has, and the capacity of your server (number of threads).
If your site has hits from many users at any time, then persistent connections may not be good for your server.
If your site has many hits, but from only a few users at a time, then persistent connections may be very good for your server. The browsers generally will open 2 or 3 concurrent connections to your server, so take this into account in your setting of MaxActiveThreads and whether you can or want to use persistent connections. If you enable persistent connections, then you will want to increase the PersistTimeout directive, which defaults to 4 seconds. You might start out at a minute or 2. You might also choose to set PersistTimeout to your average "think" time. You might have to also alter the CloseTimout directive value if anything you send to the browsers is on the large side (greater than 2K), dependent on how far away your customers are on the internet. CloseTimeout may need to be larger than its default of 60 seconds.
You will also want to increase the value use for MaxActiveThreads because a certain amount of threads will be timing out on the persistent connection, not actually running. This is probably the most important thing to do or your performance could be impacted.
A general rule of thumb for your MaxActiveThreads directive is your CPW divided by 20 (although we have seen that you can increase the number of threads on the larger iSeries systems and gain performance).
If you use too many threads, especially on smaller iSeries systems, you will negatively impact your performance because the threads thrash the processor. However, as stated above, you may be able to safely increase the value when you enable persistent connections because there will be some persistent connection timeouts.
It is rather difficult to tune this on your own. You can use your access log to determine your peak times and the number of users of your site during the peak times. However, the server does not log the persistent connection timeouts anywhere (because these are not errors or accesses), so it is difficult to determine how many persistent timeouts are occurring. The worst case scenario would be that every connection has to time out, therefore, from your peak time, you would need at most twice as many threads to maintain your performance, and then hopefully not run into the thrashing processor.
So, in conclusion, you need to make a decision on the use of persistent connections for your site. The workaround fix is simply that, a workaround. And since this is a timing window, the problems likely will never go away until the browsers get it fixed and every client workstation gets the fix. If you cannot tolerate these unpredictable results, then our recommendation is to disable persistent connections.
If you choose to allow persistent connections, then we recommend that you load one of the PTF's mentioned above. If you really think you need persistent connections, then you should strongly consider using the IBM AS400 Benchmark Center to get some help tuning the performance of the HTTP server and the iSeries. The Benchmark Center has the necessary tools to simulate workloads and the expertise to analyze and tune the iSeries appropriately. The Benchmark Center also can provide additional analysis of your applications, helping you improve their performance also.
If you choose to not allow persistent connections, just set the MaxPersistRequest directive to a value of 1. This will turn off persistent connections.
Due to a recent problem found in v4r3, we will be permanently disabling persistent connections in v4r3. This recent problem made it impossible for us to provide the workaround fix mentioned above. Without that workaround, we felt that it was best to simply disable persistent connections due to the many problems we (and you) have seen with them. This recent problem is not found in releases after v4r3, so those releases will still allow persistent connections to be configured and used.
| 
