Uploaded image for project: 'Kuali Rice Development'
  1. Kuali Rice Development
  2. KULRICE-2634

Implement support for better XML export, XML import and migration features

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: Development
    • Labels:
    • Rice Module:
      KEW

      Description

      Currently XML export is tied to lookups. When we implemented the KEW lookups for doc type, rules, etc. in Rice 1.0 we had to tie them into custom query execution instead of using the KNS's built in auto-search generation query features. As a result of this, we aren't bound by the KNS's default cap on results returned from lookups.

      It turns out this is actually a good thing for the XML export feature because it is often required to export more than 200 results (especially in the case of rules). However, this results in inconsistencies in our lookup implementations.

      What we need to look into is designing functionality which will provide better support for both XML ingestion and XML export. This would likely include:

      1) A dedicated screen for handling export of multiple types of business object data
      2) Move the XML ingester out of kew and into the core (or some other module?). Improve hooks into this feature so that even client apps can take advantage of it if they want to.
      3) Improvements to the XML ingester. Ability to create better "packaged" exports and import them instead of requiring them to handle a bunch of individual files.
      4) Once these capabilities are avaiable, retrofit the KEW lookups to properly throttle search results to be consistent with the rest of the lookups

      These are just some ideas, brainstorming and design work will be required here to flesh this out.

        Attachments

          Issue Links

            Activity

            Hide
            gtaylor Garey Taylor added a comment -

            Scott found a good site that explains how to stream xml via JAXB. This should deal with the memory issues of large xml files.

            http://www.javarants.com/2006/04/30/simple-and-efficient-xml-parsing-using-jaxb-2-0/

            Show
            gtaylor Garey Taylor added a comment - Scott found a good site that explains how to stream xml via JAXB. This should deal with the memory issues of large xml files. http://www.javarants.com/2006/04/30/simple-and-efficient-xml-parsing-using-jaxb-2-0/
            Hide
            gtaylor Garey Taylor added a comment -

            Jira that describes the tooling enhancements.

            Show
            gtaylor Garey Taylor added a comment - Jira that describes the tooling enhancements.
            Hide
            tbradford Thomas Bradford (Inactive) added a comment -

            Some thoughts on a possible approach to XML Schema Evolution

            • XML Exports are always performed using the latest version of the
              object(s) schema.
            • For import, XML Schema doesn't really require version numbers to
              provide migration. Instead, we can define a list of the historic
              schema URIs that have been defined for past versions of the object(s).
              This list can be defined either in the parsing service or possibly
              as a property of the service's spring wiring (to allow overriding).
            • For each of these schemas URIs, we associated a class that will serve
              as a converter from the immediately previous version of the schema.
              The converter is only responsible for converting that one step, and
              additional converters will be called in turn to provide each of the
              transformations required.
            • A SAX filter will do the work of calling these converters,
              transforming the XML elements as necessary and returning their new
              values and content.
            • The filter will also expose the namespace URI of the input document to
              that of the current version so that the rice instance version believes
              it has a current version of the XML document.
            • Example:

            <bean class="org.kuali.rice.test.TestImportExportService">
            <property name="filters">
            <list>
            <bean parent="ImportSchemaFilter">
            <property name="schemaUri" value="http://www.rice.org/ver1"/>
            </bean>
            <bean parent="ImportSchemaFilter">
            <property name="schemaUri" value="http://www.rice.org/ver2"/>
            <property name="schemaFilter" ref="myVer2Filter" />
            </bean>
            <bean parent="ImportSchemaFilter">
            <property name="schemaUri" value="http://www.rice.org/ver3"/>
            <property name="schemaFilter" ref="myVer3Filter" />
            </bean>
            </list>
            </property>
            </bean>

            Show
            tbradford Thomas Bradford (Inactive) added a comment - Some thoughts on a possible approach to XML Schema Evolution XML Exports are always performed using the latest version of the object(s) schema. For import, XML Schema doesn't really require version numbers to provide migration. Instead, we can define a list of the historic schema URIs that have been defined for past versions of the object(s). This list can be defined either in the parsing service or possibly as a property of the service's spring wiring (to allow overriding). For each of these schemas URIs, we associated a class that will serve as a converter from the immediately previous version of the schema. The converter is only responsible for converting that one step, and additional converters will be called in turn to provide each of the transformations required. A SAX filter will do the work of calling these converters, transforming the XML elements as necessary and returning their new values and content. The filter will also expose the namespace URI of the input document to that of the current version so that the rice instance version believes it has a current version of the XML document. Example: <bean class="org.kuali.rice.test.TestImportExportService"> <property name="filters"> <list> <bean parent="ImportSchemaFilter"> <property name="schemaUri" value="http://www.rice.org/ver1"/> </bean> <bean parent="ImportSchemaFilter"> <property name="schemaUri" value="http://www.rice.org/ver2"/> <property name="schemaFilter" ref="myVer2Filter" /> </bean> <bean parent="ImportSchemaFilter"> <property name="schemaUri" value="http://www.rice.org/ver3"/> <property name="schemaFilter" ref="myVer3Filter" /> </bean> </list> </property> </bean>
            Hide
            riceci Rice-CI User (Inactive) added a comment -

            Integrated in rice-trunk-nightly #104 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/104/)
            KULRICE-2634: Initial commit of role/permission XML import/export functionality.

            Show
            riceci Rice-CI User (Inactive) added a comment - Integrated in rice-trunk-nightly #104 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/104/ ) KULRICE-2634 : Initial commit of role/permission XML import/export functionality.
            Hide
            riceci Rice-CI User (Inactive) added a comment -

            Integrated in rice-trunk-nightly #105 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/105/)
            KULRICE-2634: removing spring file changes to allow sampleapp to start

            Show
            riceci Rice-CI User (Inactive) added a comment - Integrated in rice-trunk-nightly #105 (See http://ci.rice.kuali.org/job/rice-trunk-nightly/105/ ) KULRICE-2634 : removing spring file changes to allow sampleapp to start

              People

              • Assignee:
                Unassigned
                Reporter:
                ewestfal Eric Westfall
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: