Uploaded image for project: 'camunda BPM'
  1. camunda BPM
  2. CAM-9452

Testing Spring Boot Applications: Support application context caching

    Details

      Description

      Scenario:

      • I use the Camunda Spring Boot Starter with @EnableProcessApplication to use processes.xml for deployment
      • I write a test as follows
      package my.package;
      
      import org.camunda.bpm.engine.RuntimeService;
      import org.junit.Test;
      import org.junit.runner.RunWith;
      import org.springframework.beans.factory.annotation.Autowired;
      import org.springframework.boot.test.context.SpringBootTest;
      import org.springframework.test.context.ActiveProfiles;
      import org.springframework.test.context.junit4.SpringRunner;
      import static org.assertj.core.api.Assertions.assertThat;
      
      @RunWith(SpringRunner.class)
      @SpringBootTest
      @ActiveProfiles("test")
      public class MyTest {
      
          @Autowired
          private RuntimeService runtimeService;
      
          @Test
          public void test() {
              // do 
              runtimeService....;
          }
      }
      

      Desired behavior:

      • Spring's test runner caches application context's by default grouped by test configuration. Under the assumption that a test has no side-effects (read: does not modify the application context or any embedded resources), the context can be safely reused in another test if the test has the exact same configuration. This speeds up test runtime.

      Current behavior:

      • When using context caching and there is more than one application context (e.g. because we have tests that use two different configurations; however the process engine configuration is the same), then the creation of the second context fails
      • The problem is that we then have two process engines with the same name at the same time. The second SpringBootProcessApplication bean tries to register its engine with the MBean container and an exception is raised.
      • In consequence, users must avoid to have multiple engines with the same name (e.g. by avoiding that there are two application contexts at the same time via @DirtiesContext or by ensuring unique engine names (which does not work well with processes.xml))

        Issue Links

          Activity

          Hide
          thorben.lindhauer Thorben Lindhauer added a comment - - edited

          A couple of problematic things in these scenarios:

          • Multiple engines with the same name at the same time. Any engine-lookup-by-name APIs like ProcessEngines.getProcessEngine cannot work work correctly in this case.
          • The Spring Boot Starter makes use of the default runtime container delegate which keeps engines and process applications as MBeans identified by their name. The MBeans are managed outside of the Spring application context. This is a kind of leak, because these objects become visible to the other application context.
          • In consequence, process application deployment in one context may wrongfully be performed by the process engine of the other application context etc.
          • If we have two engines with the same name and same configuration (including same database configuration), then only duplicate checking saves us from process version increments on process application deployment. So if I configure processes.xml with disabled duplicate checking, then we get process version increments at random points in the test suite.
          • The behavior that processes are deployed for a longer time than the duration of a single test method violates the principle of test isolation.
          Show
          thorben.lindhauer Thorben Lindhauer added a comment - - edited A couple of problematic things in these scenarios: Multiple engines with the same name at the same time. Any engine-lookup-by-name APIs like ProcessEngines.getProcessEngine cannot work work correctly in this case. The Spring Boot Starter makes use of the default runtime container delegate which keeps engines and process applications as MBeans identified by their name. The MBeans are managed outside of the Spring application context. This is a kind of leak, because these objects become visible to the other application context. In consequence, process application deployment in one context may wrongfully be performed by the process engine of the other application context etc. If we have two engines with the same name and same configuration (including same database configuration), then only duplicate checking saves us from process version increments on process application deployment. So if I configure processes.xml with disabled duplicate checking, then we get process version increments at random points in the test suite. The behavior that processes are deployed for a longer time than the duration of a single test method violates the principle of test isolation.
          Hide
          thorben.lindhauer Thorben Lindhauer added a comment -

          Problems and proposed solutions:

          • Same engine name used by multiple engines => 1) Users can configure a different engine name per test configuration (and therefore application context); this already works 2) We expose an option that generates a unique engine name per Spring application context (nice to have)
          • Same H2 database used in multiple application contexts => 1) Users can configure unique datasource names and urls

          Open challenges:

          • processes.xml hardcodes a reference to a process engine, which cannot work with changing engine names => solution idea: remove the need for using processes.xml by exposing more configuration options in Spring Boot application properties
          • We should document the best practices for testing
          Show
          thorben.lindhauer Thorben Lindhauer added a comment - Problems and proposed solutions: Same engine name used by multiple engines => 1) Users can configure a different engine name per test configuration (and therefore application context); this already works 2) We expose an option that generates a unique engine name per Spring application context (nice to have) Same H2 database used in multiple application contexts => 1) Users can configure unique datasource names and urls Open challenges: processes.xml hardcodes a reference to a process engine, which cannot work with changing engine names => solution idea: remove the need for using processes.xml by exposing more configuration options in Spring Boot application properties We should document the best practices for testing
          Hide
          jan.galinski@holisticon.de Jan Galinski added a comment -

          Three short remarks:

          1.) Multiple engines are possible with camunda-engine, but the way I see it, there is currently no support for multiple engines with the spring-boot starter. So addressing this would be a different issue which should be solved a) first and b) separately

          2.) the spring context problem can be solved by using the Profile annotation with the name of the test class and a matching Active Profile ... that way, the context is valid just for one test

          3.) I already started working on a (minimal) substitution of processes.xml via application.yaml, will have to search for my local branch. Basically, I registered one or two additional steps in the ProcessApplication setup ...

          Show
          jan.galinski@holisticon.de Jan Galinski added a comment - Three short remarks: 1.) Multiple engines are possible with camunda-engine, but the way I see it, there is currently no support for multiple engines with the spring-boot starter. So addressing this would be a different issue which should be solved a) first and b) separately 2.) the spring context problem can be solved by using the Profile annotation with the name of the test class and a matching Active Profile ... that way, the context is valid just for one test 3.) I already started working on a (minimal) substitution of processes.xml via application.yaml, will have to search for my local branch. Basically, I registered one or two additional steps in the ProcessApplication setup ...
          Hide
          thorben.lindhauer Thorben Lindhauer added a comment - - edited

          Hi Jan,

          Thanks for your feedback.

          Re 1)

          That's correct for a single application context. In the case of application context caching, we have multiple application contexts and therefore multiple engines. In case those engines have the same name, this leads to various problems as described above.

          Re 2)

          That would be similar to @DirtiesContext(AFTER_CLASS), right? If that is correct, it would work fine, but would not allow for reuse of application contexts over multiple test classes, which is the goal of this ticket.

          Re 3)

          We have seen that, but we would like to choose a different way of implementation. The process application way (including deployment steps) just makes things more complicated for little gain. It is built for the shared engine case. In case of a single embedded engine, we think that deployment can be much simpler.

          Cheers,
          Thorben

          Show
          thorben.lindhauer Thorben Lindhauer added a comment - - edited Hi Jan, Thanks for your feedback. Re 1) That's correct for a single application context. In the case of application context caching, we have multiple application contexts and therefore multiple engines. In case those engines have the same name, this leads to various problems as described above. Re 2) That would be similar to @DirtiesContext(AFTER_CLASS) , right? If that is correct, it would work fine, but would not allow for reuse of application contexts over multiple test classes, which is the goal of this ticket. Re 3) We have seen that, but we would like to choose a different way of implementation. The process application way (including deployment steps) just makes things more complicated for little gain. It is built for the shared engine case. In case of a single embedded engine, we think that deployment can be much simpler. Cheers, Thorben
          Hide
          jan.galinski@holisticon.de Jan Galinski added a comment -

          1) I see
          2) correct
          3) interesting ...

          Show
          jan.galinski@holisticon.de Jan Galinski added a comment - 1) I see 2) correct 3) interesting ...
          Hide
          thorben.lindhauer Thorben Lindhauer added a comment -

          We will ping you once we have some code. Would be great to get your feedback.

          Show
          thorben.lindhauer Thorben Lindhauer added a comment - We will ping you once we have some code. Would be great to get your feedback.
          Hide
          nikola.koevski Nikola Koevski added a comment -

          Hi Jan Galinski,

          Just to give you an update on this ticket. In the end we decided not to make the proposed changes.

          After going through several iterations of possible solutions, we realized that providing a complete and compatible alternative to Process Applications for Spring Boot adds too much additional complexity. You can see the reason/explanation here: https://app.camunda.com/jira/browse/CAM-9902?focusedCommentId=102497&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-102497

          Regarding this ticket, the plan is to ensure that PA and non-PA deployments are always done to a "Default Spring Boot engine" and to provide a testing guide to ensure that users know how tests should be written to make use of application context caching when using the Camunda Spring Boot Starter.

          Best,
          Nikola

          Show
          nikola.koevski Nikola Koevski added a comment - Hi Jan Galinski , Just to give you an update on this ticket. In the end we decided not to make the proposed changes. After going through several iterations of possible solutions, we realized that providing a complete and compatible alternative to Process Applications for Spring Boot adds too much additional complexity. You can see the reason/explanation here: https://app.camunda.com/jira/browse/CAM-9902?focusedCommentId=102497&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-102497 Regarding this ticket, the plan is to ensure that PA and non-PA deployments are always done to a "Default Spring Boot engine" and to provide a testing guide to ensure that users know how tests should be written to make use of application context caching when using the Camunda Spring Boot Starter. Best, Nikola
          Hide
          nikola.koevski Nikola Koevski added a comment -

          Closing issue since all related tickets, that it depends on, have been completed. CAM-9902 was decided to not be implemented and the reason has been described in the ticket's comments.

          Show
          nikola.koevski Nikola Koevski added a comment - Closing issue since all related tickets, that it depends on, have been completed. CAM-9902 was decided to not be implemented and the reason has been described in the ticket's comments.

            People

            • Assignee:
              Unassigned
              Reporter:
              thorben.lindhauer Thorben Lindhauer
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development