Details

    • Type: Improvement Improvement
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Complete
    • Affects Version/s: None
    • Fix Version/s: 2.0.0-b6, 2.0
    • Component/s: Development
    • Security Level: Public (Public: Anyone can view)
    • Labels:
      None
    • Similar issues:
      KULRICE-1068have torque do post data load encryption in the long run?
      KULRICE-3612Implement a tool that can be used for post data load encryption, have it work better with the KIM external ids
      KULRICE-3366Cannot encrypt TAX ID with post-encrypt job
      KULRICE-7688Decrypting/Encrypting hide fields value that are not set as encrypted when click on custom button on Maintenance Document
      KULRICE-7667Decrypting/Encrypting hide fields value that are not set as encrypted when click on custom button on Maintenance Document
      KULRICE-8806Editing of value with custom editor does not invoke editor on post
      KULRICE-2028Allow custom javascript functions to be associated with Action buttons. Execution is post automatic validation if any.
      KULRICE-10857Create Automated Functional (Smoke) Tests for KRAD Labs - Encryption Utility
      KULRICE-3218Refactor encryption handling in KimEntityExternalIdentifierImpl
      KULRICE-8299Create UI for encrypting/decrypting document content
    • Rice Module:
      KNS
    • Application Requirement:
      KFS
    • KAI Review Status:
      Not Required
    • KTI Review Status:
      Not Required

      Description

      Forwarding on this change from Poonam Bhargava <bhargavp@indiana.edu> from the KFS team at IU. Please see the attached patch file which implements this change.

      Here is how it works currently:

      • Creates a backup table for the given table - "create table <table_name>_bak as select * from <table_name>"
      • Encrypts the given column in the original table
      • Drops the backup table in the end
      • In case of an exception, it rolls back the transaction and restores the original table from the backup table

      The problems we are noticing:

      • The process is very slow for huge tables, e.g. that have over a million rows.
      • If there is an exception (say network disconnection etc.), it rolls back everything, thus requiring every row to be processed again.

      Changed Approach in the customization:

      Create a back up table and add an encrypt indicator column to it.
      Perform encryption on the backup table.

      • Retrieve n rows at a time, encrypt them and commit them as a batch.
      • Update encrypt_ind to Y when a row has been encrypted.
      • In case of an error, rollback updates on this batch of n rows.
      • Back up table is dropped only if the whole process is successful - i.e. the whole backup table has been successfully encrypted and the original table has been refreshed from the backup table.

      Advantages:

      • Process is much faster. Processing time has reduced from ~10 hours to ~2.2 hours, for IU reference tables.
      • An exception doesn't rollback everything. Since the back up table is dropped only if the whole process is successful, the process can be resumed from the last successful commit.

      This is an impacting change. Please let me know if you have any questions or concerns.

        Issue Links

          Activity

          Hide
          Heather Elyea added a comment -

          The IU-Rice jira is closed, has this been contributed to Rice-foundation yet?

          Show
          Heather Elyea added a comment - The IU-Rice jira is closed, has this been contributed to Rice-foundation yet?
          Hide
          Eric Westfall added a comment -

          Shannon, can you integrate this change in with 2.0.0-b5 once it's out? Thanks!

          Show
          Eric Westfall added a comment - Shannon, can you integrate this change in with 2.0.0-b5 once it's out? Thanks!
          Hide
          Shannon Hess added a comment -

          yep, I'll plan on doing this as soon as 2.0.0-b5 is out.

          Show
          Shannon Hess added a comment - yep, I'll plan on doing this as soon as 2.0.0-b5 is out.
          Hide
          Shannon Hess added a comment -

          PostDataLoadEncryptionSpringBeans.xml has the following bean, which refers to class PostDataLoadEncryptionDaoJdbc. However, that class no longer exists and I can't find any jiras that mention the removal of this class. Is it OK to add this class back or was it removed for a particular reason? To me it seems like it shouldn't have been deleted.

          <bean id="postDataLoadEncryptionDao" parent="platformAwareDaoJdbc" class="org.kuali.rice.krad.dao.jdbc.PostDataLoadEncryptionDaoJdbc" />

          Show
          Shannon Hess added a comment - PostDataLoadEncryptionSpringBeans.xml has the following bean, which refers to class PostDataLoadEncryptionDaoJdbc. However, that class no longer exists and I can't find any jiras that mention the removal of this class. Is it OK to add this class back or was it removed for a particular reason? To me it seems like it shouldn't have been deleted. <bean id="postDataLoadEncryptionDao" parent="platformAwareDaoJdbc" class="org.kuali.rice.krad.dao.jdbc.PostDataLoadEncryptionDaoJdbc" />
          Hide
          Shannon Hess added a comment -

          Ended up creating /development-tools/src/main/java/org/kuali/rice/krad/dao/jdbc/PostDataLoadEncryptionDaoJdbc.java

          Show
          Shannon Hess added a comment - Ended up creating /development-tools/src/main/java/org/kuali/rice/krad/dao/jdbc/PostDataLoadEncryptionDaoJdbc.java
          Hide
          Samuel Wong added a comment -

          The service class PostDataLoadEncryptionService is removed in the latest rice 2.0 branch, and so we cannot enable the step in KFS end.

          Show
          Samuel Wong added a comment - The service class PostDataLoadEncryptionService is removed in the latest rice 2.0 branch, and so we cannot enable the step in KFS end.
          Hide
          Eric Westfall added a comment -

          It looks like this class is now in rice-development-tools module under the package "org.kuali.rice.devtools.pdle".

          Show
          Eric Westfall added a comment - It looks like this class is now in rice-development-tools module under the package "org.kuali.rice.devtools.pdle".

            People

            • Assignee:
              Shannon Hess
              Reporter:
              James Bennett
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Structure Helper Panel