Panic! At the optimization (for pineappl > 1)

[scarlehoff]

I'm getting a panic at optimization time (I believe, looking at the exception, see below).
This only happens for versions of pineappl above > 1

I have uploaded a grid that fails to optimize (even from the CLI), just rename .zip to .lz4

I've tested the following combinations:

pineappl-0.8.7-py313h5e81bce_4.conda - Works
pineappl-1.3.0-py313h7ef63f5_0.conda - Does not work
pineappl 1.3.0 # installed with the script, does not work

The stack backtrace below is for 1.3.0

thread '<unnamed>' panicked at pineappl/src/interpolation.rs:34:9:
internal error: entered unreachable code
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at core/src/panicking.rs:221:5:
panic in a function that cannot unwind
stack backtrace:
   0:     0x7fcdcf554bf9 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::hc002032c745ec399
   1:     0x7fcdcf4c260b - core::fmt::write::h78425ec9962d4be7
   2:     0x7fcdcf525cbe - std::io::Write::write_fmt::ha00b146e89c56092
   3:     0x7fcdcf555ff2 - std::sys::backtrace::BacktraceLock::print::h59ceb186f016850e
   4:     0x7fcdcf555e5b - std::panicking::default_hook::{{closure}}::h9fcfa010b5ecd19c
   5:     0x7fcdcf5565b6 - std::panicking::rust_panic_with_hook::h0a4b940661a26423
   6:     0x7fcdcf556122 - std::panicking::begin_panic_handler::{{closure}}::h72160a9fdca8fdc8
   7:     0x7fcdcf5560b9 - std::sys::backtrace::__rust_end_short_backtrace::hb6826ea75cd9c94b
   8:     0x7fcdcf5560a4 - rust_begin_unwind
   9:     0x7fcdcf4c59d4 - core::panicking::panic_nounwind_fmt::he104e461f7e92412
  10:     0x7fcdcf4c5a3c - core::panicking::panic_nounwind::hef5dfaaaad24ed17
  11:     0x7fcdcf4c59e4 - core::panicking::panic_cannot_unwind::h44cd3738aca63805
  12:     0x7fcdcf4ab1f2 - pineappl_grid_optimize
  13:     0x7fcdcfe849d6 - __pineappl_mod_MOD_write_grid_pine
  14:     0x7fcdcfe8bf39 - __histograms_mod_MOD_writetofile_hist
  15:     0x7fcdcff251bd - __vegas_mod_MOD_vegas
  16:     0x7fcdcffae22e - vegaswrapper_
  17:     0x55d57e1e4d5c - MAIN__
  18:     0x55d57e1e2061 - main
  19:     0x7fcdcf81a083 - __libc_start_main
                               at /build/glibc-B3wQXB/glibc-2.31/csu/../csu/libc-start.c:308:16
  20:     0x55d57e1e209e - _start
  21:                0x0 - <unknown>
thread caused non-unwinding panic. aborting.

Program received signal SIGABRT: Process abort signal.

Backtrace for this error:
#0  0x7fcdcf83908f in ???
        at /build/glibc-B3wQXB/glibc-2.31/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
#1  0x7fcdcf83900b in __GI_raise
        at ../sysdeps/unix/sysv/linux/raise.c:51
#2  0x7fcdcf818858 in __GI_abort
        at /build/glibc-B3wQXB/glibc-2.31/stdlib/abort.c:79

~$ pineappl subgrids fail_optimize.lz4 --x=1
o b  c                                                                                                                                                                                                                                                        x1
-+--+-+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 0  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 1  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 2  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 3  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 4  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 5  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 6  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 7  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 8  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 9  0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7
0 10 0 1.000e0, 9.309e-1, 8.628e-1, 7.956e-1, 7.296e-1, 6.648e-1, 6.015e-1, 5.398e-1, 4.799e-1, 4.222e-1, 3.669e-1, 3.144e-1, 2.651e-1, 2.195e-1, 1.780e-1, 1.411e-1, 1.091e-1, 8.228e-2, 6.048e-2, 4.341e-2, 3.052e-2, 2.109e-2, 1.438e-2, 9.699e-3, 6.496e-3, 4.329e-3, 2.874e-3, 1.903e-3, 1.259e-3, 8.314e-4, 5.488e-4, 3.621e-4, 2.388e-4, 1.575e-4, 1.038e-4, 6.844e-5, 4.511e-5, 2.974e-5, 1.960e-5, 1.292e-5, 8.517e-6, 5.614e-6, 3.700e-6, 2.439e-6, 1.608e-6, 1.060e-6, 6.984e-7, 4.604e-7, 3.034e-7, 2.000e-7

fail_optimize.zip

[scarlehoff]

RUST_BACKTRACE=full pineappl write fail_optimize.lz4 --optimize test.lz4

thread 'main' panicked at /Users/juacrumar/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pineappl-1.3.0/src/interpolation.rs:34:9:
internal error: entered unreachable code
stack backtrace:
   0:        0x104327418 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::hf35f9734f9a29483
   1:        0x104345f98 - core::fmt::write::h60ec6633daab7b35
   2:        0x1043237b4 - std::io::Write::write_fmt::hc29709fdab2e34e2
   3:        0x1043272cc - std::sys::backtrace::BacktraceLock::print::hca95bffd78053951
   4:        0x104328c90 - std::panicking::default_hook::{{closure}}::h357ed4fbef22679d
   5:        0x104328ae8 - std::panicking::default_hook::h0a4e133b151d5758
   6:        0x104329730 - std::panicking::rust_panic_with_hook::h557a23724a5de839
   7:        0x104329324 - std::panicking::begin_panic_handler::{{closure}}::h269cace6208fef05
   8:        0x1043278c8 - std::sys::backtrace::__rust_end_short_backtrace::h5be0da278f3aaec7
   9:        0x104329028 - __rustc[de2ca18b4c54d5b8]::rust_begin_unwind
  10:        0x10435ee58 - core::panicking::panic_fmt::h477ff48eff31ffa4
  11:        0x10435eec8 - core::panicking::panic::h3c0d0c5e47386e71
  12:        0x10414e228 - pineappl::interpolation::applgrid::fx2::h29cdcaf2ef9a8cfe
  13:        0x1041341a0 - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold::hdfad29f5606fc449
  14:        0x104133768 - <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::hf8bee456c9c3df79
  15:        0x104157ab4 - pineappl::interpolation::Interp::node_values::hdd6fc81df8d76407
  16:        0x10412fd74 - <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::h2406d6576bf6bb6b
  17:        0x1041512d8 - <pineappl::subgrid::InterpSubgridV1 as pineappl::subgrid::Subgrid>::indexed_iter::h252f1eb8145d8b52
  18:        0x1041503d8 - <pineappl::subgrid::ImportSubgridV1 as core::convert::From<&pineappl::subgrid::SubgridEnum>>::from::h2213dc7c7162645e
  19:        0x10413e25c - pineappl::grid::Grid::optimize_using::h7c7207c9e91c4f30
  20:        0x1040eaa90 - <pineappl_cli::write::Opts as pineappl_cli::Subcommand>::run::hea3d13c6067728e2
  21:        0x10403f038 - pineappl::main::h34092fe213d03268
  22:        0x10403e620 - std::sys::backtrace::__rust_begin_short_backtrace::h70524ec46be09021
  23:        0x10403e608 - std::rt::lang_start::{{closure}}::hc67ebf8dadce2588
  24:        0x10431c9c4 - std::rt::lang_start_internal::hdb28e94b6865fa11
  25:        0x10403f218 - _main

[scarlehoff]

Adding the y to the unreachable as suggested:

internal error: entered unreachable code: 3.7515209639507225

This is done with the current tip of the repository, I just cloned it.
The first failing commit is c0b83dd

[felixhekhorn]

The x grid listed above seems pretty much to be the default grid, which is tested here:

pineappl/pineappl/src/interpolation.rs

Line 450 in 41e8af5

let x_reference = [

Given that this test passes fine in the CI: do we test both backward and forward transformation? i.e. x->y and y->x? I'm not sure about that and the error might be only in one of the directions

[Radonirinaunimi]

Addressed by #380.

[Radonirinaunimi]

For the record, this also appeared in MadGraph https://answers.launchpad.net/mg5amcnlo/+question/823822.

[cschwan]

Thanks for keeping track of this! 👍