Python 3.14 refcount semantics and problems with resize

[bmcfee]

I'm just flagging this for you so you're aware, though it's likely to be fixed upstream of you (ie in numpy) anyway.

I've been seeing test failures in librosa on 3.14 environments lately, and after discussing with numpy devs, traced it back to the following issue numpy/numpy#30991 wherein the resize operation causes problems with memory reference counting on python 3.14 (but not earlier).

Note: this is a rather sneaky bug, as it only pops up for me when bundling resample calls within a vectorization like np.apply_along_axis (as we do in librosa to handle multichannel).

Looking into the python-samplerate code, it appears that resize is generally used to make a truncated view of an over-allocated output array:

python-samplerate/src/samplerate.cpp

Lines 184 to 188 in 235d720

	// create a shorter view of the array
	if ((size_t)src_data.output_frames_gen < new_size) {
	out_shape[0] = src_data.output_frames_gen;
	output.resize(out_shape);
	}

It's probably more hassle than it's worth to rewrite these bindings to avoid view resize. (Unless libsamplerate exposes a function to pre-compute output lengths?) It may also work (and is probably safe) to force refcheck=False into the resize calls, but that seems a little hacky IMO.

[mattip]

It's probably more hassle than it's worth to rewrite these bindings to avoid view resize

NumPy dev piling on here. After a naive look at the code it seems like newsize could be adjusted before allocating output, rather than after. But there are probably things I don't understand.

In general, np.resize in my opinion is something to be avoided. It can lead to memory fragmentation.

[bmcfee]

In general, np.resize in my opinion is something to be avoided. It can lead to memory fragmentation.

👍 to that, though I think in this case it's not actually that bad. I didn't write this code or the underlying library that it's wrapping, but I have implemented these algorithms independently in the past. The code in question is doing a specific kind of interpolation over the input array, and due to the vagaries of floating point non-associativity, it's easy to end up in off-by-one errors when computing output lengths by rounding. As such, it's safer to over-allocate and trim (view-trim), and the fragmentation is relatively small (ie one sample, or 4-8 bytes typically).

At least, I'm reasonably confident that's how it plays out in practice, though I could be missing some details.

[mattip]

Ahh, then it might work to return a view output[:newsize], if it only overallocates by a row or two, rather than trying to resize it.

[ngoldbaum]

FWIW, I think the use here is unsafe, and the error being raised on 3.14 is correctly pointing out an unsafe pattern.

Unfortuantely on 3.13 and older, it's impossible to tell the difference between an array created by an extension with one reference and a uniquely referenced array created by a Python function. The old check was not stringent enough to catch issues in C extensions. But on 3.14 due to changes in reference counting semantics for function locals, we now check for truly uniquely referenced arrays, and that catches the code in python-samplerate.

I think in practice the chance of issues here is small, but I think the error is probably pointing out a real possible source of unsafety that's been undetected until now.

[tuxu]

Thanks for raising the issue and discussing here. I think it's best to return a view, which is also how v0.1.0 worked.

[fakufaku]

Thanks for raising the issue and chiming in here.
I've created a fix that changes the resize for a view in #36 .
@tuxu Please review and approve the change if you have a chance.