Array of const broken in 32bit, Stack issue in 64bit

[TetzkatLipHoka]

The recent update broke the Array of const support in 32bit in some way.

This script worked fine on my previous build (I think about 85 commits ago)

uses
  SysUtils;

var
  i, j : Integer;
begin
  Help3.Clear;  
  for j := 0 to Help1.Count-1 do
    begin
    for i := 0 to Help2.Count-1 do
      Help3.Add( Format( Help2[ i ], [ Trim( Help1[ j ] ), Trim( Help1[ j ] ) ] ) );    
    Help3.Add( '' );
    end;
end.

Now it crashes on 32bit with script exception when trying to call "format".
On 64bit Format works fine but accessing Help1 inline with j > 0 returns an empty string
Ex:
Help[x]=" OK"
0 : Format( 'Test%s', [ Help1[0] ] ) -> "Test OK"
1 : Format( 'Test%s', [ Help1[1] ] ) -> "Test"

Visualisation
Working

32bit, broken

[martijnlaan]

Thanks. Do you have a more complete example? Help1/2/3 are not defined, and Help1/2 initialization is also missing.

Also, which compiler is this and are you using InvokeCall.inc, or x86/64.inc?

BTW, you could try to use Git bisect to quickly isolate the commit which causes the problem.

[TetzkatLipHoka]

Ofc, whatever you need^^

I build using Delphi 10.3 Rio.
USEINVOKECALL is defined by

{$IFNDEF FPC}
...
  {$IFDEF DELPHI23UP}
  {$IFNDEF AUTOREFCOUNT}
  {$IFNDEF PS_USECLASSICINVOKE}
    {$DEFINE USEINVOKECALL}
  {$ENDIF}
  {$ENDIF}
  {$ENDIF}

in uPSRuntime

Help1-3 are SynEdits placed on Help-Forms as seen in the Screenshots.

While making the example I made some further tests ...
This would be the TStringList equivalent - this works as expected tho (32/64bit).

uses
  SysUtils;

var
  StrL1,
  StrL2,
  StrL3 : TStringList;

  i, j : Integer;
  S    : String;
begin
  StrL1 := TStringList.Create;
  StrL2 := TStringList.Create;
  StrL3 := TStringList.Create;
  
  StrL1.Add( 'Jane' );
  StrL1.Add( 'Amber' );
  StrL1.Add( 'Miku' );
  StrL1.Add( 'Sanae' );
  
  StrL2.Add( '%s likes pizza' );

  StrL3.Clear;
  for j := 0 to StrL1.Count-1 do
    begin
    for i := 0 to StrL2.Count-1 do
      StrL3.Add( Format( StrL2[ i ], [ Trim( StrL1[ j ] ), Trim( StrL1[ j ] ) ] ) );  
    end;    
    
  StrL1.free;
  StrL2.free;
  StrL3.free;    
end.

The Issue seems to be caused by access to the SynEdit.Lines by Stack access.
The SynEdit.Lines are registered for direct access like this:

  ...
  fPSScript                             := TPSScriptDebugger.Create( self );
  fPSScript.OnCompile                   := PSScriptCompile;
  fPSScript.OnExecute                   := PSScriptExecute;  
  ...

procedure TFrPascalScript.PSScriptCompile( Sender: TPSScript );
var
  i : Integer;
begin
  for i := Low( fHelpWindows ) to High( fHelpWindows ) do
    Sender.AddRegisteredVariable( 'Help' + IntToStr( i+1 ), 'TStrings' );
end;

procedure TFrPascalScript.PSScriptExecute( Sender: TPSScript );
var
  i : Integer;
begin
  for i := Low( fHelpWindows ) to High( fHelpWindows ) do
    fPSScript.SetVarToInstance( 'Help' + IntToStr( i+1 ), fHelpWindows[ i ].Edit.Lines );
end;

This works fine (32/64bit):

var
  S, S2 : String;
begin
...
  S := Help2[ i ];
  S2 := Trim( Help1[ j ] );  
  S := Format( S, [ S2, S2 ] );

while this causes the exception (on 32bit) or empty strings (if j > 0) (on 64bit):

var
  S, S2 : String;
begin
...
  S := Format( Help2[ i ], [ Trim( Help1[ j ] ), Trim( Help1[ j ] ) ] );

[martijnlaan]

Thanks.

• Does it happen when you use x86.inc/x64.inc instead of InvokeCall.inc by defining PS_USECLASSICINVOKE?
• Does it happen if you move the StrL1 etc to Delphi and use AddRegisteredVariable/SetVarToInstance for them? In other words, is using SynEdit really relevant?
• Can the example be reduced? It seems the for loops aren't relevant since your last example doesn't have any (but it also doesnt say what value i and j are)? Also it seems Help3 is irrelevant since the last example doesnt use it? In fact, can it be reduced to use only Help1 or only Help2? Or doesn it really need 2 lists to reproduce the issue? And are the Trim calls needed?
• Have you tried using git bisect? Just checkout the latest commit and then enter this on the command line: git bisect start bad. Then choose a commit in the past until you find one that works again and then enter git bisect good. It will then auto pick a commit in between for you to test and to enter either git bisect good or git bisect bad. This should quickly end at the commit introducing the issue. (Note: you can abort a bisect session by entering git bisect reset.)

Asking these things because the examples you posted so far don't allow me te produce your issue.

[TetzkatLipHoka]

• PS_USECLASSICINVOKE works fine for 32/64bit
• Global TStringLists with AddRegisteredVariable/SetVarToInstance also works fine on 32/64bit
• Loops are not required, calling format like this is sufficient:

    Help3.Add( Format( Help2[ 0 ], [ Trim( Help1[ 1 ] ), Trim( Help1[ 1 ] ) ] ) );

This will cause an exception on 32bit and return empty strings for Help1[ 1 ] on 64bit, Help1[ 0 ] works.

Iam not much into git mechanics, so no I didnt use bisect

[TetzkatLipHoka]

Issue is resolved - for both 32 and 64bit - by reverting:
btString: aValues := aValues + [TValue.From<TbtString>(pTbtString(aValue^.dta)^)];
to
btString: aValues := aValues + [TValue.From<String>(pTbtString(aValue^.dta)^)];
in InvokeCall.inc:71 (TbtString->String)

[martijnlaan]

Thanks for figuring that out. However, the From and To look the same to me? Also, InvokeCall.inc stays 'btrecord:' for me on line 71.

[TetzkatLipHoka]

Haiya ... GitEditor .... without Quote-Block the relevant part becomes invisible.
In my - updated - code its Line 71, possible it moved.
Look for TValue.From<TbtString> its unique in the File.

Out of context: Is there a reason my pull-request is ignored?

[martijnlaan]

Thank you, I see the difference now. I don't understand the change though, TbtString is the correct type for btString, and String isn't. Why is this change needed? And why only that line? There are three other lines related to btString in InvokeCall.inc. Does the issue exist when using the source code from this repo as-is, so without any of your changes?

I don't think anyone from Remobjects is watching pull requests or issues, so in that sense everything here is ignored by them.

I'm not from Remobjects, but do have commit access and try to help when there's a PR or issue which can be reviewed and has a clear benefit. The problem with PR #294 is that it changes many independent things at once without an explanation of the benefits of each change, which makes it hard to review as a whole. Ideally, PRs should be small and focused.

[TetzkatLipHoka]

Yes, I tested against your last code with minimal changes to get my stuff working (Adding exports from SysUtils for Format-function)
Regarding why the change is needed I have this - AI - explanation ;)

TbtString is defined as AnsiString on Delphi 2009+ (see uPSUtils.pas). The issue is that TValue.From<TbtString> (i.e. TValue.From<AnsiString>) creates a TValue with RTTI TypeInfo of AnsiString. When this TValue later flows into TValueArrayToArrayOfConst (for array of const / TVarRec construction), Delphi's RTTI marshals it as an AnsiString pointer. However, SysUtils.Format and other RTL functions on Delphi 2009+ expect TVarRec string entries to be UnicodeString (string). Using TValue.From<String> ensures the TValue carries UnicodeString TypeInfo, which TValueArrayToArrayOfConst then correctly maps to vtUnicodeString in the TVarRec. With AnsiString TypeInfo it gets mapped to vtAnsiString, and Format on modern Delphi either misinterprets the pointer (crash on 32-bit) or reads stale/empty data (wrong result on 64-bit, especially when the source is a temporary like an indexed property result).

Regarding the other three btString lines:

The other btString occurrences in InvokeCall.inc serve different purposes:
Line 69 (btString: aValues := aValues + [...]) — this is the line in question, used when building the TValue array for value parameters, which then feeds into TValueArrayToArrayOfConst for array of const. This is the only path where the TypeInfo mismatch matters.
Line 144-152 (inside the open array/btString branch) — this copies string data into a contiguous TArray<tbtString> and then pushes a raw Pointer to that array. The TValue there is TValue.From<Pointer>, so the string's TypeInfo isn't used for dispatch.
Line 465 (result handling) — this reads back the return value via .AsString and assigns it to tbtString(res.dta^). The implicit conversion from UnicodeString to AnsiString happens at assignment, so it works correctly either way.

So only line 69 is affected because it's the only place where the TValue's TypeInfo is used to determine how the string gets marshaled into a TVarRec.

Regarding the PR, well they once were single, more structured commits until I had to remerge with your latest changes/additions.
Iam currently - partially - merging the pullt fork/PR149 adding (switchable) Unicode-Mode for btString and missing PChar types.
Once iam done with that I will - try to - break the PR into multiple pushes with more explanation again ...

[martijnlaan]

Thank you for the explanation. I can't commit this change based on it though. Changing the type from AnsiString to UnicodeString for all string parameters just to make Format work is a workaround, not a fix. It may simply be that InvokeCall.inc doesn't support passing AnsiString to functions like Format.

Note that the stable branch excludes InvokeCall.inc entirely, and I would recommend using that branch instead. Or define PS_USECLASSICINVOKE.

Regarding the PR: the earlier PR (#265) had the same issue, so this is not related to recent changes. The number of commits and how they are structured inside the PR is also not relevant, it's the total set of changes that matters.