[sgkalluri]

Hello there,

Ever since we have used ProcessMaker we have been facing with an issue that we have not understood. It goes like this.

a] Time between clicking an entry in Inbox to appearance of Case # on the top left corner of the screen = 1 to 2 secs.

b] Time taken between start of first trigger before form load to end of last trigger before form load = 1 sec or so, depending upon the trigger complexity.

c] Time taken between completion of triggers before form load and start of form load on the user screen = 2 to 40 secs or even more, depending on the size of data stored.

d] Time taken between start of form load to end of form load = 1 to 5 secs or more, depending on # of fields and any MySQL scripts that are run before the user sees the screen.

The one that concerns us is the time c] mentioned above. On my Windows 10, 12 GB RAM laptop, where I do some process development, I see that the time c] varies from 1-2 secs for a 13K data size (as judged from the APP_DATA field) to 32 secs for a 188K data size.

This time c] seems to depend only on the amount of data stored, and maybe on the # of PMT report tables defined. But, the estimate is that it depends more on the data stored.

The time that c] takes on the production server, which is a Linux, Centos 7, 16GB RAM server, is probably half of what is observed on my Windows 10 laptop.

I had conducted an experiment to see if this issue is related in some way to the interaction between PHP and MySQL. I created a dummy PMT table and then wrote a trigger to insert records into MySQL. I did not see a problem there. The record insertion was happening quickly.

My perception is that this is related in some way to the way PM handles saving of data from the session to MySQL. The reason I say this is that I think that data is saved from the session into the APP_DATA field as well as into the PMT Reports tables when all the triggers are executed before the form loads, or when a save action is given from the Dynaform, or when all triggers are executed after form unload, or when the triggers of other sections like Assignment, Before Routing and After Routing are executed.

This is of course, just based on my observation and perception.

Is this normal behaviour?

Is there a method to reduce time c]?

Best wishes,
Satya Gopal

[amosbatto]

If you have a lot of data in your APPLICATION.APP_DATA field, then it will really slow down ProcessMaker, because that data has to be read from MySQL and written to MySQL with each change of a Dynaform field or a variable in a trigger.

I recommend removing case variables that you no longer need to speed up cases. For example, if you have a large variable to populate the list of options in a dropdown, you could use this trigger after the Dynaform to unset the variable:

Code: Select all

unset(@=dropdownOptionsList);

[sgkalluri]

Thanks a lot, Amos.

I'll have to change the design of the Dynaforms and Triggers then. I may have to have multiple Dynaforms, each containing only the variables to be changed. The same with the triggers.

Will update later on.

Thanks again.
SGK

[sgkalluri]

Hello there,

I have conducted an experiment with a fairly complex process. I used a dummy Dynaform that had just one variable. I also did not add any triggers. So, no change in values of variables via triggers. The only action maybe that system variables like APPLICATION, TASK, USER_LOGGED etc. would get updated.

Here are the results...

Code: Select all

Case Id    # of Variables Stored in APP_DATA    Size of APP_DATA (bytes)    Approx Time Taken in Secs for Time c]
1579       60                                   5907                        2-3
1575       57                                   11532                       4-5
1533       204                                  139816                      49-50
1577       191                                  215092                      58-59

One can see that as the Size of APP_DATA increases, the loading time c] (as referenced in the first post) increases exponentially. The # of Variables may not be causative factor.

This result surprised me a bit, as I expected the time to load to be a couple of seconds or so as the Dynaform used was a single variable one and as there were no triggers, so no real change in variables.

Is there some other factor that affects loading time c], maybe updation of APP_HISTORY table or something like that, and whether this would impact the loading time c].

Best wishes,
SGK

[amosbatto]

MySQL should be faster than that. I wonder if PHP's unserialize() function is that slow. Increasing the size of the memory_limit in your php.ini file might help if PHP is memory constrained.

I don't think that APP_HISTORY is slowing you down when you load a case, because it writes new records after you submit a Dynaform or Input Document.

In your env.ini file, make sure that you don't have these lines:
debug = 1
debug_sql = 1
They will kill performance.

Of course, using the Enterprise Edition will make your case lists much faster. However, I don't think that the Enterprise Edition will improve loading times of cases with a large APP_DATA.

[sgkalluri]

Thanks for the directions, Amos.

So, I do not have the statements you mentioned in the env.ini file.

PHP memory_limit was experimented with 512M, 1024M and 2048M, with no significant change in loading time c].

I then conducted another experiment. I went to a dummy process and wrote a dummy trigger that would use MySQL to select APP_DATA from the respective application records of the cases. I then unserialized the retrieved APP_DATA. Here are the results...

Case # 1579 (data size 5907 bytes)
MySQL query execution time to retrieve APP_DATA = 0.0037 secs
PHP Unserialize function execution on APP_DATA = 0.0001 secs

Case # 1577 (data size 215092 bytes)
MySQL query execution time to retrieve APP_DATA = 0.0042 secs
PHP Unserialize function execution on APP_DATA = 0.0018 secs

So, MySQL queries and PHP Unserialize function seem to be working fine.

I then conducted another experiment. I copied the retrieved APP_DATA into a dummy variable in this dummy process, knowing that the APP_DATA size of the dummy case would increase corresponding to the APP_DATA size of the case in question (1579 or 1577), and then checked whether the loading time c] for the dummy case would increase.

The result was that the loading time c] did not increase with smaller or larger APP_DATA size. The loading time c] was between 2-3 secs for the smaller size case, 1579, as well as the larger size case, 1577.

So, the root cause seems to be somewhere else...

Will conduct a few more experiments and revert.

Best wishes,
SGK

[amosbatto]

Try turning off logging:
viewtopic.php?f=47&t=730020

Writing those log files will take a long time if you have a large APPLICATION.APP_DATA.

[sgkalluri]

I'll definitely try that, Amos.

In the meantime, I conducted one more experiment. I wondered if this loading time is related to the existence of PM Report Tables for the complex process in question.

There are 7 PM Report Tables for this complex process. I then deleted these PM Report Tables and checked what would be the loading time for the simple Dynaform that I had created initially. And, the loading time c] remains between 2-3 secs for the smaller case 1579 as well as the larger case 1577! I introduced the PM Report Tables to replicate the issue, and found that the larger case 1577 again took 58-59 secs to load.

So, the presence of PM Report Tables does contribute significantly to the loading time c] of a process! I am not sure why though.

If there is no workaround, then I may have to create PM Tables instead of PM Report Tables, and then update them using normal MySQL queries for the purpose of reports. That may reduce loading time c]. I'll have to try this option out. It may take a couple of days for me.

Best wishes,
SGK

[amosbatto]

Interesting. I wouldn't expect PM to write to Report Tables when loading a case. The PM codes does a lot of strange things that I don't expect.