Kuali Rice Development
  1. Kuali Rice Development
  2. KULRICE-2634

Implement support for better XML export, XML import and migration features

    Details

    • Type: Improvement Improvement
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: Development
    • Labels:
    • Similar issues:
      KULRICE-5041Migrate xml import/export framework to the Rice core
      KULRICE-2507Add support for fromDate and toDate to Rule XML export and import
      KULRICE-1130improve performance of xml ingestion
      KULRICE-2241Add XML Export support to KNS lookups
      KULRICE-2287Add XML Export support to KNS inquiries
      KULRICE-4332 Build xml export support for Groups screen
      KULRICE-12461Library - CSV/XML/XLS export not working
      KULRICE-4073XML Transform Export
      KULRICE-750Rule XML import and export does not contain or consider active indicator
      KULRICE-4624Reinvestigation into applying nice formatting and rendering for exported XML
    • Rice Module:
      KEW

      Description

      Currently XML export is tied to lookups. When we implemented the KEW lookups for doc type, rules, etc. in Rice 1.0 we had to tie them into custom query execution instead of using the KNS's built in auto-search generation query features. As a result of this, we aren't bound by the KNS's default cap on results returned from lookups.

      It turns out this is actually a good thing for the XML export feature because it is often required to export more than 200 results (especially in the case of rules). However, this results in inconsistencies in our lookup implementations.

      What we need to look into is designing functionality which will provide better support for both XML ingestion and XML export. This would likely include:

      1) A dedicated screen for handling export of multiple types of business object data
      2) Move the XML ingester out of kew and into the core (or some other module?). Improve hooks into this feature so that even client apps can take advantage of it if they want to.
      3) Improvements to the XML ingester. Ability to create better "packaged" exports and import them instead of requiring them to handle a bunch of individual files.
      4) Once these capabilities are avaiable, retrofit the KEW lookups to properly throttle search results to be consistent with the rest of the lookups

      These are just some ideas, brainstorming and design work will be required here to flesh this out.

        Issue Links

          Activity

          Hide
          Garey Taylor added a comment -

          Scott found a good site that explains how to stream xml via JAXB. This should deal with the memory issues of large xml files.

          http://www.javarants.com/2006/04/30/simple-and-efficient-xml-parsing-using-jaxb-2-0/

          Show
          Garey Taylor added a comment - Scott found a good site that explains how to stream xml via JAXB. This should deal with the memory issues of large xml files. http://www.javarants.com/2006/04/30/simple-and-efficient-xml-parsing-using-jaxb-2-0/
          Hide
          Garey Taylor added a comment -

          Jira that describes the tooling enhancements.

          Show
          Garey Taylor added a comment - Jira that describes the tooling enhancements.
          Hide
          Thomas Bradford (Inactive) added a comment -

          Some thoughts on a possible approach to XML Schema Evolution

          • XML Exports are always performed using the latest version of the
            object(s) schema.
          • For import, XML Schema doesn't really require version numbers to
            provide migration. Instead, we can define a list of the historic
            schema URIs that have been defined for past versions of the object(s).
            This list can be defined either in the parsing service or possibly
            as a property of the service's spring wiring (to allow overriding).
          • For each of these schemas URIs, we associated a class that will serve
            as a converter from the immediately previous version of the schema.
            The converter is only responsible for converting that one step, and
            additional converters will be called in turn to provide each of the
            transformations required.
          • A SAX filter will do the work of calling these converters,
            transforming the XML elements as necessary and returning their new
            values and content.
          • The filter will also expose the namespace URI of the input document to
            that of the current version so that the rice instance version believes
            it has a current version of the XML document.
          • Example:

          <bean class="org.kuali.rice.test.TestImportExportService">
          <property name="filters">
          <list>
          <bean parent="ImportSchemaFilter">
          <property name="schemaUri" value="http://www.rice.org/ver1"/>
          </bean>
          <bean parent="ImportSchemaFilter">
          <property name="schemaUri" value="http://www.rice.org/ver2"/>
          <property name="schemaFilter" ref="myVer2Filter" />
          </bean>
          <bean parent="ImportSchemaFilter">
          <property name="schemaUri" value="http://www.rice.org/ver3"/>
          <property name="schemaFilter" ref="myVer3Filter" />
          </bean>
          </list>
          </property>
          </bean>

          Show
          Thomas Bradford (Inactive) added a comment - Some thoughts on a possible approach to XML Schema Evolution XML Exports are always performed using the latest version of the object(s) schema. For import, XML Schema doesn't really require version numbers to provide migration. Instead, we can define a list of the historic schema URIs that have been defined for past versions of the object(s). This list can be defined either in the parsing service or possibly as a property of the service's spring wiring (to allow overriding). For each of these schemas URIs, we associated a class that will serve as a converter from the immediately previous version of the schema. The converter is only responsible for converting that one step, and additional converters will be called in turn to provide each of the transformations required. A SAX filter will do the work of calling these converters, transforming the XML elements as necessary and returning their new values and content. The filter will also expose the namespace URI of the input document to that of the current version so that the rice instance version believes it has a current version of the XML document. Example: <bean class="org.kuali.rice.test.TestImportExportService"> <property name="filters"> <list> <bean parent="ImportSchemaFilter"> <property name="schemaUri" value="http://www.rice.org/ver1"/> </bean> <bean parent="ImportSchemaFilter"> <property name="schemaUri" value="http://www.rice.org/ver2"/> <property name="schemaFilter" ref="myVer2Filter" /> </bean> <bean parent="ImportSchemaFilter"> <property name="schemaUri" value="http://www.rice.org/ver3"/> <property name="schemaFilter" ref="myVer3Filter" /> </bean> </list> </property> </bean>
          Hide
          Rice-CI User (Inactive) added a comment -

          Integrated in rice-trunk-nightly #104 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/104/)
          KULRICE-2634: Initial commit of role/permission XML import/export functionality.

          Show
          Rice-CI User (Inactive) added a comment - Integrated in rice-trunk-nightly #104 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/104/ ) KULRICE-2634 : Initial commit of role/permission XML import/export functionality.
          Hide
          Rice-CI User (Inactive) added a comment -

          Integrated in rice-trunk-nightly #105 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/105/)
          KULRICE-2634: removing spring file changes to allow sampleapp to start

          Show
          Rice-CI User (Inactive) added a comment - Integrated in rice-trunk-nightly #105 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/105/ ) KULRICE-2634 : removing spring file changes to allow sampleapp to start

            People

            • Assignee:
              Unassigned
              Reporter:
              Eric Westfall
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Structure Helper Panel