The function took 5.05 s to compute.\n+The function took 5.16 s to compute.\n \n The transformed data are:\n [ 0.49671415 -0.1382643 0.64768854 1.52302986 -0.23415337 -0.23413696\n 1.57921282 0.76743473 -0.46947439 0.54256004]\n
\n \n At the first call, the results will be cached. Therefore, the computation\n@@ -168,45 +168,45 @@\n data_trans = transformer.transform(data)\n end = time.time()\n \n print('\\nThe function took {:.2f} s to compute.'.format(end - start))\n print('\\nThe transformed data are:\\n {}'.format(data_trans))\n
The function took 5.05 s to compute.\n+The function took 5.11 s to compute.\n \n The transformed data are:\n [ 0.49671415 -0.1382643 0.64768854 1.52302986 -0.23415337 -0.23413696\n 1.57921282 0.76743473 -0.46947439 0.54256004]\n
\n \n start = time.time()\n data_trans = transformer.transform(data)\n end = time.time()\n \n print('\\nThe function took {:.2f} s to compute.'.format(end - start))\n print('\\nThe transformed data are:\\n {}'.format(data_trans))\n
\n \n-The function took 0.02 s to compute.\n+The function took 0.06 s to compute.\n \n The transformed data are:\n [ 0.49671415 -0.1382643 0.64768854 1.52302986 -0.23415337 -0.23413696\n 1.57921282 0.76743473 -0.46947439 0.54256004]\n
\n \n As expected, the second call to the transform method load the results\n which have been cached.
\n \n \n Clean up cache directory\u00b6
\n memory.clear(warn=False)\n
\n \n-Total running time of the script: (0 minutes 15.237 seconds)
\n+Total running time of the script: (0 minutes 15.777 seconds)
\n \n
\n First round - caching the data\n-Elapsed time for the entire processing: 4.37 s\n+Elapsed time for the entire processing: 4.53 s\n
\n \n By using 2 workers, the parallel processing gives a x2 speed-up compared to\n the sequential case. By executing again the same process, the intermediate\n results obtained by calling costly_compute_cached will be loaded from the\n cache instead of executing the function.
\n \n \n Second round - reloading from the cache\n-Elapsed time for the entire processing: 0.01 s\n+Elapsed time for the entire processing: 0.12 s\n
\n \n \n \n Reuse intermediate checkpoints\u00b6
\n \n \n
\n Reusing intermediate checkpoints\n-Elapsed time for the entire processing: 0.01 s\n+Elapsed time for the entire processing: 0.10 s\n
\n \n The processing time only corresponds to the execution of the max\n function. The internal call to costly_compute_cached is reloading the\n results from the cache.
\n \n \n Clean-up the cache folder\u00b6
\n memory.clear(warn=False)\n
\n \n-Total running time of the script: (0 minutes 12.403 seconds)
\n+Total running time of the script: (0 minutes 12.778 seconds)
\n \n Create result generator with return_as='generator'...\n Accumulate results:......................................................................................................................................................\n All tasks completed and reduced successfully.\n-Peak memory usage: 122.35MB\n+Peak memory usage: 118.18MB\n We can then report the memory usage accross time of the two runs using the\n MemoryMonitor.
\nIn the first case, as the results accumulate in res, the memory grows\n linearly and it is freed once the accumulator_sum function finishes.
In the second case, the results are processed by the accumulator as soon as\n@@ -200,15 +200,15 @@\n plt.show()\n
![\"parallel](\"../_images/sphx_glr_parallel_generator_001.png\")
It is important to note that with return_as="generator", the results are\n still accumulated in RAM after computation. But as we asynchronously process\n them, they can be freed sooner. However, if the generator is not consumed\n the memory still grows linearly.
Total running time of the script: (0 minutes 43.027 seconds)
\n+Total running time of the script: (1 minutes 8.945 seconds)
\n