Details

    • Type: Feature Request
    • Status: Done
    • Priority: L3 - Default
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 2.5.0, 2.5.0-alpha1
    • Component/s: backend, frontend
    • Labels:
      None
    • PM Priority:
      40

      Description

      AT:

      • the number of datapoints that a report can evaluate is restricted to 1000
      • if a report evaluation would contains more than the maximum allowed databpoints, then only the first 1000 entries are shown and a warning is displayed to the user similar to the one that already exists in the raw data report
      • changes in the rest api are adjusted in the documentation
      • there is documentation in the user guide about this behavior

      Context:
      When the user creates a report, he usually selects a view, groupby and visualize as and the report is displayed. However, there are scenarios where there are a lot of datapoints to display, e.g. if you grouped by StartDate: Hour with data from several years or grouped by a variable with a lot of distinct values.

      Depending on the visualization and browser, this might create problems where the page appears to become unresponsive while the huge dataload is transferred and processed.

      Also there is a back-end dimension to this: if there are too many datapoints, Elasticsearch will reject to evaluate a query in the future. Also it can crash Optimize/Elasticsearch if the queries evaluate too many datapoints.

        Activity

        sebastian.stamm Sebastian Stamm created issue -
        johannes.heinemann Johannes Heinemann made changes -
        Field Original Value New Value
        Workflow Camunda Optimize - 2.0 [ 54578 ] Camunda Optimize - 2.4 [ 64762 ]
        johannes.heinemann Johannes Heinemann made changes -
        Labels next_release
        johannes.heinemann Johannes Heinemann made changes -
        Component/s backend [ 13653 ]
        Component/s frontend [ 13654 ]
        felix.mueller Felix Müller made changes -
        Fix Version/s 2.5 [ 15364 ]
        felix.mueller Felix Müller made changes -
        Rank Ranked higher
        felix.mueller Felix Müller made changes -
        Rank Ranked higher
        felix.mueller Felix Müller made changes -
        Summary Warn user when Report contains a lot of datapoints Give warning of too large reports
        johannes.heinemann Johannes Heinemann made changes -
        Fix Version/s 2.5.0 [ 15385 ]
        Fix Version/s 2.5 [ 15364 ]
        felix.mueller Felix Müller made changes -
        Rank Ranked higher
        sebastian.bathke Sebastian Bathke made changes -
        Labels next_release current_release next_release
        sebastian.bathke Sebastian Bathke made changes -
        Labels current_release next_release current_release
        johannes.heinemann Johannes Heinemann made changes -
        Description When the user creates a report, he usually selects a view, groupby and visualize as and the report is displayed. However, there are scenarios where there are a lot of datapoints to display, e.g. if you grouped by StartDate: Hour with data from several years or grouped by a variable with a lot of distinct values.

        Depending on the visualization and browser, this might create problems where the page appears to become unresponsive while the huge dataload is transferred and processed.

        My proposal would be to not display the report result in the report edit mode right away if the result contains more than a certain number of datapoints. Instead, we should display a warning to the user like "The report you are about to create contains 72.195 datapoints. Do you want to create a start date or variable filter?" The filter mentions can be links that directly open the corresponding filter modal. There should also be a "Display anyway" button.

        This warning should not appear in the Report (non-edit) view or in Dashboards.
        *AT:*
        * the number of datapoints that a report can evaluate is restricted to 1000
        * if a report evaluation would contains more than the maximum allowed databpoints, then only the first 1000 entries are shown and a warning is displayed to the user similar to the one that already exists in the raw data report
        * changes in the rest api are adjusted in the documentation
        * there is documentation in the user guide about this behavior

        *Context:*
        When the user creates a report, he usually selects a view, groupby and visualize as and the report is displayed. However, there are scenarios where there are a lot of datapoints to display, e.g. if you grouped by StartDate: Hour with data from several years or grouped by a variable with a lot of distinct values.

        Depending on the visualization and browser, this might create problems where the page appears to become unresponsive while the huge dataload is transferred and processed.

        Also there is a back-end dimension to this: if there are too many datapoints, Elasticsearch will reject to evaluate a query in the future. Also it can crash Optimize/Elasticsearch if the queries evaluate too many datapoints.
        sebastian.bathke Sebastian Bathke made changes -
        Status Open [ 1 ] In Specification [ 10000 ]
        johannes.heinemann Johannes Heinemann made changes -
        Status In Specification [ 10000 ] In Development [ 10312 ]
        sebastian.bathke Sebastian Bathke made changes -
        Status In Development [ 10312 ] Done [ 10010 ]
        Resolution Done [ 10000 ]
        felix.mueller Felix Müller made changes -
        PM Priority 40
        michael.wagner Michael Wagner made changes -
        Fix Version/s 2.5.0-alpha1 [ 15391 ]
        michael.wagner Michael Wagner made changes -
        Labels current_release

          People

          • Assignee:
            Unassigned
            Reporter:
            sebastian.stamm Sebastian Stamm
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: