Keeping track of the performance

[guilhermebodin]

We have added a few benchmarks and can run them every time a new release is out. This is the first one.
edit in the PR #82 we changed some benchmarks. This is the run with the updated benchmarks

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface1.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [4076af6c] JuMP v1.0.0
  [b8f27783] MathOptInterface v1.4.0
  [0ce4ce61] ParametricOptInterface v0.3.0

variables on a MOIU.Model.
  227.000 μs (63 allocations: 658.20 KiB)
variables on a POI.Optimizer.
  417.500 μs (137 allocations: 1.01 MiB)
parameters on a POI.Optimizer.
  171.500 μs (100 allocations: 608.34 KiB)
parameters and variables on a POI.Optimizer.
  305.400 μs (158 allocations: 1.16 MiB)
alternating parameters and variables on a POI.Optimizer.
  317.300 μs (152 allocations: 1.00 MiB)
SAF constraint with variables on a MOIU.Model.
  2.940 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.270 ms (752 allocations: 31.55 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  18.362 ms (4184 allocations: 97.07 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.178 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  5.891 ms (952 allocations: 46.82 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  29.708 ms (6112 allocations: 110.91 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  18.699 ms (3412 allocations: 50.43 MiB)
SAF objective with variables on a MOIU.Model.
  2.478 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.956 ms (737 allocations: 31.54 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  20.071 ms (3758 allocations: 89.42 MiB)
SQF objective with variables on a MOIU.Model.
  6.929 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  7.418 ms (937 allocations: 46.82 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  24.425 ms (5676 allocations: 110.89 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  17.687 ms (2976 allocations: 50.40 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  26.699 ms (4244 allocations: 97.47 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  39.061 ms (5810 allocations: 93.81 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  10.178 s (12006793 allocations: 37.68 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  34.103 ms (3777 allocations: 50.83 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  6.278 s (4456821 allocations: 37.50 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  47.142 ms (5027 allocations: 54.72 MiB)

[guilhermebodin]

Version 0.3.1

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [b8f27783] MathOptInterface v1.1.2
  [0ce4ce61] ParametricOptInterface v0.3.1

variables on a MOIU.Model.
  224.600 μs (62 allocations: 658.17 KiB)
variables on a POI.Optimizer.
  334.600 μs (132 allocations: 922.06 KiB)
parameters on a POI.Optimizer.
  187.100 μs (102 allocations: 608.23 KiB)
parameters and variables on a POI.Optimizer.
  253.000 μs (152 allocations: 948.11 KiB)
alternating parameters and variables on a POI.Optimizer.
  250.800 μs (146 allocations: 791.62 KiB)
SAF constraint with variables on a MOIU.Model.
  2.994 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.368 ms (747 allocations: 31.44 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  17.693 ms (4474 allocations: 96.90 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.342 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  5.897 ms (947 allocations: 46.72 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  25.680 ms (6507 allocations: 110.75 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  17.041 ms (3807 allocations: 50.26 MiB)
SAF objective with variables on a MOIU.Model.
  2.469 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.805 ms (732 allocations: 31.44 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  14.868 ms (3752 allocations: 89.19 MiB)
SQF objective with variables on a MOIU.Model.
  3.768 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  4.308 ms (932 allocations: 46.71 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  24.763 ms (5770 allocations: 110.67 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  17.447 ms (3070 allocations: 50.18 MiB)
Update parameters in SAF constraint with parameters on a POI.Optimizer.
  513.919 ms (8102 allocations: 101.55 MiB)
Update parameters in SAF objective with parameters on a POI.Optimizer.
  41.914 ms (7388 allocations: 93.92 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer. M = 10
  3.176 s (1103585 allocations: 20.56 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  833.899 ms (78239 allocations: 59.39 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer. M = 10
  607.143 ms (201319 allocations: 3.75 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  49.738 ms (6705 allocations: 54.84 MiB)

[guilhermebodin]

Version 0.3.2 edit in the PR #82 we changed some benchmarks. This is the run with the updated benchmarks

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface1.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [b8f27783] MathOptInterface v1.1.2
  [0ce4ce61] ParametricOptInterface v0.3.2

variables on a MOIU.Model.
  222.800 μs (62 allocations: 658.17 KiB)
variables on a POI.Optimizer.
  331.500 μs (132 allocations: 922.06 KiB)
parameters on a POI.Optimizer.
  164.700 μs (102 allocations: 608.23 KiB)
parameters and variables on a POI.Optimizer.
  252.400 μs (152 allocations: 948.11 KiB)
alternating parameters and variables on a POI.Optimizer.
  255.100 μs (146 allocations: 791.62 KiB)
SAF constraint with variables on a MOIU.Model.
  2.986 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.384 ms (747 allocations: 31.44 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  10.571 ms (2274 allocations: 62.05 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.497 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  6.006 ms (947 allocations: 46.72 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  13.472 ms (3207 allocations: 58.49 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  14.896 ms (2707 allocations: 24.15 MiB)
SAF objective with variables on a MOIU.Model.
  2.555 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.812 ms (732 allocations: 31.44 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  8.561 ms (1552 allocations: 54.35 MiB)
SQF objective with variables on a MOIU.Model.
  4.199 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  4.394 ms (932 allocations: 46.71 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  12.517 ms (2470 allocations: 58.41 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  14.587 ms (1970 allocations: 24.07 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  17.572 ms (2334 allocations: 62.45 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  32.669 ms (5184 allocations: 58.96 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  6.273 s (2007570 allocations: 37.41 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  30.297 ms (4469 allocations: 24.64 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  7.185 s (2008095 allocations: 37.42 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  55.520 ms (5601 allocations: 28.61 MiB)

[guilhermebodin]

Version 0.3.3

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface1.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [b8f27783] MathOptInterface v1.1.2
  [0ce4ce61] ParametricOptInterface v0.3.3

variables on a MOIU.Model.
  235.400 μs (63 allocations: 658.20 KiB)
variables on a POI.Optimizer.
  343.300 μs (138 allocations: 922.00 KiB)
parameters on a POI.Optimizer.
  209.300 μs (114 allocations: 757.28 KiB)
parameters and variables on a POI.Optimizer.
  280.700 μs (162 allocations: 841.03 KiB)
alternating parameters and variables on a POI.Optimizer.
  279.400 μs (156 allocations: 684.55 KiB)
SAF constraint with variables on a MOIU.Model.
  2.884 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.204 ms (753 allocations: 31.44 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  7.330 ms (2284 allocations: 61.95 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.283 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  5.812 ms (953 allocations: 46.72 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  10.619 ms (3217 allocations: 58.38 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  10.991 ms (2717 allocations: 24.05 MiB)
SAF objective with variables on a MOIU.Model.
  2.442 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.782 ms (738 allocations: 31.44 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  5.839 ms (1562 allocations: 54.24 MiB)
SQF objective with variables on a MOIU.Model.
  3.764 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  4.318 ms (938 allocations: 46.71 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  10.023 ms (2480 allocations: 58.30 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  10.898 ms (1980 allocations: 23.97 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  8.922 ms (2324 allocations: 61.99 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  17.651 ms (5172 allocations: 58.43 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  6.864 s (2007560 allocations: 37.41 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  15.421 ms (4459 allocations: 24.18 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  5.670 s (2008085 allocations: 37.42 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  24.501 ms (5590 allocations: 28.15 MiB)

[guilhermebodin]

Version 0.3.4

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface1.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [b8f27783] MathOptInterface v1.1.2
  [0ce4ce61] ParametricOptInterface v0.3.4

variables on a MOIU.Model.
  227.000 μs (63 allocations: 658.20 KiB)
variables on a POI.Optimizer.
  362.700 μs (142 allocations: 922.84 KiB)
parameters on a POI.Optimizer.
  222.600 μs (118 allocations: 758.12 KiB)
parameters and variables on a POI.Optimizer.
  297.900 μs (166 allocations: 841.88 KiB)
alternating parameters and variables on a POI.Optimizer.
  280.400 μs (160 allocations: 685.39 KiB)
SAF constraint with variables on a MOIU.Model.
  3.400 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.308 ms (757 allocations: 31.44 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  8.978 ms (2494 allocations: 61.96 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.445 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  6.031 ms (957 allocations: 46.72 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  10.851 ms (3418 allocations: 58.39 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  11.574 ms (2918 allocations: 24.06 MiB)
SAF objective with variables on a MOIU.Model.
  2.503 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.932 ms (742 allocations: 31.44 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  6.163 ms (1566 allocations: 54.24 MiB)
SQF objective with variables on a MOIU.Model.
  3.876 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  4.314 ms (942 allocations: 46.71 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  10.092 ms (2484 allocations: 58.30 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  11.061 ms (1984 allocations: 23.97 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  10.034 ms (2506 allocations: 62.00 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  17.465 ms (2171 allocations: 58.12 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  6.772 s (2007726 allocations: 37.41 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  15.477 ms (4625 allocations: 24.19 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  6.667 s (2005084 allocations: 37.42 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  26.373 ms (2589 allocations: 27.85 MiB)

[guilhermebodin]

Version 0.3.5

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface1.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [b8f27783] MathOptInterface v1.1.2
  [0ce4ce61] ParametricOptInterface v0.3.5

variables on a MOIU.Model.
  227.500 μs (63 allocations: 658.20 KiB)
variables on a POI.Optimizer.
  355.000 μs (146 allocations: 923.45 KiB)
parameters on a POI.Optimizer.
  207.200 μs (122 allocations: 758.73 KiB)
parameters and variables on a POI.Optimizer.
  273.800 μs (170 allocations: 842.48 KiB)
alternating parameters and variables on a POI.Optimizer.
  259.400 μs (164 allocations: 686.00 KiB)
SAF constraint with variables on a MOIU.Model.
  2.924 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.194 ms (761 allocations: 31.45 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  8.158 ms (3005 allocations: 62.03 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.299 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  5.803 ms (961 allocations: 46.72 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  11.360 ms (3422 allocations: 58.39 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  10.954 ms (2922 allocations: 24.06 MiB)
SAF objective with variables on a MOIU.Model.
  2.482 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.841 ms (746 allocations: 31.44 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  6.079 ms (1570 allocations: 54.24 MiB)
SQF objective with variables on a MOIU.Model.
  3.836 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  4.385 ms (946 allocations: 46.71 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  10.057 ms (2488 allocations: 58.30 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  10.948 ms (1988 allocations: 23.97 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  9.833 ms (3023 allocations: 62.07 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  17.193 ms (2175 allocations: 58.12 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  6.532 s (2007730 allocations: 37.41 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  14.858 ms (4629 allocations: 24.19 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  5.579 s (2005088 allocations: 37.42 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  26.587 ms (2593 allocations: 27.85 MiB)

[guilhermebodin]

version 0.4.0

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface1.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [4076af6c] JuMP v1.1.0
  [b8f27783] MathOptInterface v1.3.0
  [0ce4ce61] ParametricOptInterface v0.4.0

variables on a MOIU.Model.
  225.200 μs (63 allocations: 658.20 KiB)
variables on a POI.Optimizer.
  338.900 μs (146 allocations: 923.45 KiB)
parameters on a POI.Optimizer.
  209.300 μs (122 allocations: 758.73 KiB)
parameters and variables on a POI.Optimizer.
  274.500 μs (170 allocations: 842.48 KiB)
alternating parameters and variables on a POI.Optimizer.
  264.400 μs (164 allocations: 686.00 KiB)
SAF constraint with variables on a MOIU.Model.
  3.016 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.236 ms (761 allocations: 31.45 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  8.732 ms (2413 allocations: 61.91 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.466 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  5.947 ms (961 allocations: 46.72 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  12.959 ms (3126 allocations: 58.34 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  11.370 ms (2626 allocations: 24.01 MiB)
SAF objective with variables on a MOIU.Model.
  2.656 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.943 ms (746 allocations: 31.44 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  6.253 ms (1570 allocations: 54.24 MiB)
SQF objective with variables on a MOIU.Model.
  4.064 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  4.576 ms (946 allocations: 46.71 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  10.058 ms (2188 allocations: 50.66 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  11.285 ms (1788 allocations: 23.96 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  12.022 ms (2418 allocations: 61.92 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  17.849 ms (1675 allocations: 54.25 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  8.665 s (2507233 allocations: 37.43 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  15.604 ms (3928 allocations: 24.04 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  6.952 s (2004888 allocations: 37.41 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  24.927 ms (1893 allocations: 23.97 MiB)

[guilhermebodin]

Version 0.4.1

Pkg status:
      Status `C:\Users\guilhermebodin\Documents\Github\ParametricOptInterface1.jl\benchmark\Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [4076af6c] JuMP v1.0.0
  [b8f27783] MathOptInterface v1.4.0
  [0ce4ce61] ParametricOptInterface v0.4.1

variables on a MOIU.Model.
  232.300 μs (63 allocations: 658.20 KiB)
variables on a POI.Optimizer.
  356.200 μs (146 allocations: 923.45 KiB)
parameters on a POI.Optimizer.
  216.400 μs (122 allocations: 758.73 KiB)
parameters and variables on a POI.Optimizer.
  289.500 μs (170 allocations: 842.48 KiB)
alternating parameters and variables on a POI.Optimizer.
  278.300 μs (164 allocations: 686.00 KiB)
SAF constraint with variables on a MOIU.Model.
  2.866 ms (677 allocations: 31.19 MiB)
SAF constraint with variables on a POI.Optimizer.
  3.196 ms (761 allocations: 31.45 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  8.732 ms (2413 allocations: 61.91 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  5.055 ms (877 allocations: 46.46 MiB)
SQF constraint with variables on a POI.Optimizer.
  5.703 ms (961 allocations: 46.72 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  10.855 ms (3126 allocations: 58.34 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  10.981 ms (2626 allocations: 24.01 MiB)
SAF objective with variables on a MOIU.Model.
  2.482 ms (562 allocations: 31.18 MiB)
SAF objective with variables on a POI.Optimizer.
  2.843 ms (746 allocations: 31.44 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  6.239 ms (1570 allocations: 54.24 MiB)
SQF objective with variables on a MOIU.Model.
  4.025 ms (762 allocations: 46.45 MiB)
SQF objective with variables on a POI.Optimizer.
  4.471 ms (946 allocations: 46.71 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  9.296 ms (2188 allocations: 50.66 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  11.283 ms (1788 allocations: 23.96 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  9.366 ms (2418 allocations: 61.92 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  16.727 ms (1675 allocations: 54.25 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  7.949 s (2507233 allocations: 37.43 GiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  15.630 ms (3928 allocations: 24.04 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  6.800 s (2004888 allocations: 37.41 GiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  24.666 ms (1893 allocations: 23.97 MiB)

[odow]

Some very good updates, but also a few ones with more allocations.

julia> include("benchmark/run_benchmarks.jl")
Pkg status:
Status `~/git/jump-dev/ParametricOptInterface/benchmark/Project.toml`
  [6e4b80f9] BenchmarkTools v1.6.3
  [4076af6c] JuMP v1.29.4
  [b8f27783] MathOptInterface v1.49.0
  [0ce4ce61] ParametricOptInterface v0.14.2 `..`

variables on a MOIU.Model.
  64.083 μs (54 allocations: 614.88 KiB)
variables on a POI.Optimizer.
  70.375 μs (95 allocations: 617.59 KiB)
parameters on a POI.Optimizer.
  425.541 μs (20117 allocations: 1.37 MiB)
parameters and variables on a POI.Optimizer.
  251.000 μs (10151 allocations: 1.17 MiB)
alternating parameters and variables on a POI.Optimizer.
  249.625 μs (10143 allocations: 970.23 KiB)
SAF constraint with variables on a MOIU.Model.
  1.109 ms (868 allocations: 31.87 MiB)
SAF constraint with variables on a POI.Optimizer.
  1.192 ms (1125 allocations: 31.88 MiB)
SAF constraint with variables and parameters on a POI.Optimizer.
  3.940 ms (13127 allocations: 63.72 MiB)
SQF constraint with variables on a MOIU.Model{Float64}.
  1.813 ms (1068 allocations: 50.63 MiB)
SQF constraint with variables on a POI.Optimizer.
  1.952 ms (1325 allocations: 50.64 MiB)
SQF constraint with product of variables and parameters on a POI.Optimizer.
  17.099 ms (16051 allocations: 88.25 MiB)
SQF constraint with product of parameters on a POI.Optimizer.
  7.257 ms (14351 allocations: 26.55 MiB)
SAF objective with variables on a MOIU.Model.
  707.333 μs (654 allocations: 31.86 MiB)
SAF objective with variables on a POI.Optimizer.
  1.290 ms (1395 allocations: 47.51 MiB)
SAF objective with variables and parameters on a POI.Optimizer.
  3.840 ms (12752 allocations: 79.33 MiB)
SQF objective with variables on a MOIU.Model.
  1.343 ms (1054 allocations: 50.62 MiB)
SQF objective with variables on a POI.Optimizer.
  1.819 ms (1695 allocations: 75.65 MiB)
SQF objective with product of variables and parameters on a POI.Optimizer.
  13.693 ms (15476 allocations: 91.35 MiB)
SQF objective with product of parameters on a POI.Optimizer.
  7.680 ms (13976 allocations: 39.03 MiB)
Update parameters in SAF constraint with variables and parameters on a POI.Optimizer.
  6.915 ms (13132 allocations: 63.72 MiB)
Update parameters in SAF objective with variables and parameters on a POI.Optimizer.
  13.907 ms (12852 allocations: 79.33 MiB)
Update parameters in SQF constraint with product of variables and parameters on a POI.Optimizer.
  848.905 ms (1017356 allocations: 136.95 MiB)
Update parameters in SQF constraint with product of parameters on a POI.Optimizer.
  15.380 ms (14756 allocations: 26.60 MiB)
Update parameters in SQF objective with product of variables and parameters on a POI.Optimizer.
  842.784 ms (516576 allocations: 126.17 MiB)
Update parameters in SQF objective with product of parameters on a POI.Optimizer.
  22.252 ms (14477 allocations: 39.08 MiB)