KFS Request
  1. KFS Request
  2. KFSMI-5916

SessionDocumentService can be a performance bottleneck

    Details

    • Type: Bug Fix Bug Fix
    • Status: Rice Roadblock Rice Roadblock
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: TBD
    • Sub-Committee:
      Tech / QA / PM
    • Impacted Modules:
      System

      Description

      It appears that the SessionDocumentService is a major source of performance issues, especially for large documents. For every request the user makes on the document, the entire document is serialized and transferred to and from the database.

      Some of the performance may be due to the size of the connection pool, but across two servers it should have been sufficient for the number of users who were testing. It really seemed to have to do with the sheer amount of data being transferred. On some maintenance documents (which is all we were testing), the serialized object graph was over a megabyte.

      By stubbing out the service to remove the feature altogether, we were able to greatly improve server performance. That may not be the ultimate solution, but it meets UCDs needs at the moment.

      Another option would be to use a shared file system for such a store, since that would be faster to access than via the database connections.

        Issue Links

          Activity

          Hide
          Ailish Byrne added a comment -

          i'm confused - i thought it only went to the db to get the doc when it wasn't in "true session". this is really disappointing, since one of the main goals here was performance

          Show
          Ailish Byrne added a comment - i'm confused - i thought it only went to the db to get the doc when it wasn't in "true session". this is really disappointing, since one of the main goals here was performance
          Hide
          Jonathan Keller added a comment -

          The problem is that it has to save the document each time you post. In order to do that, it first has to pull the object from the database (OJB - version number), update the serialized object, and re-persist it. That would be the major benefit of moving to the file system - no need to retrieve.

          Also, the tests really showed how I/O bound this system is. During our load test, operations on maintenance documents (after the initial load) took an average of 16 seconds. But only used on average 111ms of CPU time. It's something that you can't easily see with yourkit because the instrumentation slows the system down to such a point that it's more in line with the I/O speeds. (And this is with the database on the same private subnet as the KFS servers.)

          Show
          Jonathan Keller added a comment - The problem is that it has to save the document each time you post. In order to do that, it first has to pull the object from the database (OJB - version number), update the serialized object, and re-persist it. That would be the major benefit of moving to the file system - no need to retrieve. Also, the tests really showed how I/O bound this system is. During our load test, operations on maintenance documents (after the initial load) took an average of 16 seconds. But only used on average 111ms of CPU time. It's something that you can't easily see with yourkit because the instrumentation slows the system down to such a point that it's more in line with the I/O speeds. (And this is with the database on the same private subnet as the KFS servers.)
          Hide
          Jonathan Keller added a comment -

          Oh - and if memory serves, the SessionDocument persistence was not for performance. It was to restore the functionality we theoretically lost when we moved from request-based forms. That a user's unsaved document could survive a server restart or bounce to a different node in a server cluster.

          We should probably make use of this service an option, since small institutions with a single server probably don't need this functionality as much. I'm also going to look into the Tomcat session clustering and see what that buys us.

          Show
          Jonathan Keller added a comment - Oh - and if memory serves, the SessionDocument persistence was not for performance. It was to restore the functionality we theoretically lost when we moved from request-based forms. That a user's unsaved document could survive a server restart or bounce to a different node in a server cluster. We should probably make use of this service an option, since small institutions with a single server probably don't need this functionality as much. I'm also going to look into the Tomcat session clustering and see what that buys us.

            People

            • Reporter:
              Jonathan Keller
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Structure Helper Panel