Kuali Rice Development
  1. Kuali Rice Development
  2. KULRICE-4287

Intermittent issues with KSB service lookup while running on embedded mode.

    Details

    • Type: Bug Fix Bug Fix
    • Status: Closed Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: KC Release 3.0, 1.0.3
    • Fix Version/s: KC Release 3.0, 1.0.3
    • Component/s: Development
    • Labels:
      None
    • Similar issues:
      KULRICE-6636Incorrect KEW service references while running in embedded mode
      KULRICE-6783Incorrect service reference to RoutingReportService while running Routing report in embedded mode
      KULRICE-6788KSB SOAPFaultException while attempting to insert a BO Note (while running in embedded mode)
      KULRICE-6763Another instance of incorrect kew service reference in embedded mode
      KULRICE-7169Rice object lookup quickfinders not working when related module is in EMBEDDED mode
      KULRICE-6917LocationModuleConfiguration is not available for embedded clients
      KULRICE-10335Startup issues with Rice 2.4 in embedded mode
      KULRICE-6683Add a "THIN" client run mode to KEW, KIM, and KSB modules
      KULRICE-2758Split KNS services for local vs. embedded
      KULRICE-7015enRuleAttributeService must be defined locally in KEW EMBEDDED mode
    • Rice Module:
      KSB
    • Application Requirement:
      KC

      Description

      see linked issue for details

        Issue Links

          Activity

          Hide
          Jeremy Hanson added a comment -

          Slowly. I worked on it for a while and had some services quarantining, but for some reason it broke the basic functionality of KSB. I've assigned it to Emmerson because I didn't have the time needed to dedicate to the jira at the time.

          Show
          Jeremy Hanson added a comment - Slowly. I worked on it for a while and had some services quarantining, but for some reason it broke the basic functionality of KSB. I've assigned it to Emmerson because I didn't have the time needed to dedicate to the jira at the time.
          Hide
          Eric Westfall added a comment -

          I'm going to attempt to tackle this one. I think the first thing I'm going to need to do here is try and make sense of the relationship between RemoteResourceServiceLocatorImpl and RemotedServiceRegistryImpl. There is an unfortunate lack of separation of concerns between these two classes. It's very difficult to describe what the responsibility of each of these classes is.

          In general, I think what we need for the KSB is the following:

          1) A service (which should ultimately be able invoked remotely) which allows for querying the service registry and for publishing services to the registry.
          2) A service which is deployed on each KSB client which interacts with the Service Registry service and maintains client-side proxies to services published on the bus. This should probably include both local and remote services. This will essentially maintain the client-sides copy of the service registry.
          3) A service used by the above service to periodically scan the registry for differences between the central registry and the client-side registry.
          4) A ResourceLoader implementation which interfaces with the above service.

          I'd like to make this refactoring right now but I'm not sure that we have time for it. Ultimately, the problem here is that the BusClientFailureProxy has no way to perform logic for quarantining of particular endpoints since it doesn't know the specific other endpoints that are available. Instead it marks a service as "dead" and then goes back to the RemoteResourceServiceLocator and fetches the service again to re-invoke it.

          I think what needs to happen here is that the BusClientFailureProxy really needs to be responsible for knowing all of the services that are available for a particular endpoint, and then orchestrating quarantining of those endpoints when necessary, and performing intelligent failover when that is necessary. Another possiblity is to have an external service handle logic surrounding availability of service endpoints and then delegate to that to determine what a suitable endpoint is to try. I'll keep thinking through this a bit and try and come up with a design that is simple and implementable within the current KSB setup.

          Show
          Eric Westfall added a comment - I'm going to attempt to tackle this one. I think the first thing I'm going to need to do here is try and make sense of the relationship between RemoteResourceServiceLocatorImpl and RemotedServiceRegistryImpl. There is an unfortunate lack of separation of concerns between these two classes. It's very difficult to describe what the responsibility of each of these classes is. In general, I think what we need for the KSB is the following: 1) A service (which should ultimately be able invoked remotely) which allows for querying the service registry and for publishing services to the registry. 2) A service which is deployed on each KSB client which interacts with the Service Registry service and maintains client-side proxies to services published on the bus. This should probably include both local and remote services. This will essentially maintain the client-sides copy of the service registry. 3) A service used by the above service to periodically scan the registry for differences between the central registry and the client-side registry. 4) A ResourceLoader implementation which interfaces with the above service. I'd like to make this refactoring right now but I'm not sure that we have time for it. Ultimately, the problem here is that the BusClientFailureProxy has no way to perform logic for quarantining of particular endpoints since it doesn't know the specific other endpoints that are available. Instead it marks a service as "dead" and then goes back to the RemoteResourceServiceLocator and fetches the service again to re-invoke it. I think what needs to happen here is that the BusClientFailureProxy really needs to be responsible for knowing all of the services that are available for a particular endpoint, and then orchestrating quarantining of those endpoints when necessary, and performing intelligent failover when that is necessary. Another possiblity is to have an external service handle logic surrounding availability of service endpoints and then delegate to that to determine what a suitable endpoint is to try. I'll keep thinking through this a bit and try and come up with a design that is simple and implementable within the current KSB setup.
          Hide
          Eric Westfall added a comment -

          Ok, I've been doing some analysis on this one and came up with the following document:

          https://wiki.kuali.org/x/-ok_Eg

          The full scope of the work here could be quite significant and probably a bit risky for 1.0.3. Instead, I'd like to focus on the short-term fixes as outlined in the document.

          I've already dealt with one of these regarding the way that KIM was caching null and bad service endpoints. Another suggestion is to improve a bit how the BusClientFailureProxy deals with bad services.

          Beyond that, I'd like to understand more about the reasons why you have to end up clearing your KSB tables so much. For example, what specific problems you are running into which are triggering you to clear them out. The only thing I really see mentioned in KCIRB-649 is the NullPointerException coming from the attempts to use a KimTypeService (which is the problem that should already be fixed).

          Show
          Eric Westfall added a comment - Ok, I've been doing some analysis on this one and came up with the following document: https://wiki.kuali.org/x/-ok_Eg The full scope of the work here could be quite significant and probably a bit risky for 1.0.3. Instead, I'd like to focus on the short-term fixes as outlined in the document. I've already dealt with one of these regarding the way that KIM was caching null and bad service endpoints. Another suggestion is to improve a bit how the BusClientFailureProxy deals with bad services. Beyond that, I'd like to understand more about the reasons why you have to end up clearing your KSB tables so much. For example, what specific problems you are running into which are triggering you to clear them out. The only thing I really see mentioned in KCIRB-649 is the NullPointerException coming from the attempts to use a KimTypeService (which is the problem that should already be fixed).
          Hide
          Eric Westfall added a comment -

          Modified the BusClientFailureProxy so that it resets the target service when it fails over. This should hopefully address part of the problem here in the failure proxies for the KSB.

          Show
          Eric Westfall added a comment - Modified the BusClientFailureProxy so that it resets the target service when it fails over. This should hopefully address part of the problem here in the failure proxies for the KSB.
          Hide
          Eric Westfall added a comment -

          I'm going to go ahead and resolve this in light of the fixes that I checked in for this earlier today. Please reopen if there are other issues we need to address. Thanks.

          Show
          Eric Westfall added a comment - I'm going to go ahead and resolve this in light of the fixes that I checked in for this earlier today. Please reopen if there are other issues we need to address. Thanks.

            People

            • Assignee:
              Eric Westfall
              Reporter:
              Travis Schneeberger
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1 week
                1w
                Remaining:
                Remaining Estimate - 1 week
                1w
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Structure Helper Panel