How to keep a pointer to a Dynamic Array valid after resizing the array?

Question

I have a dynamic array, and I need to maintain a pointer to it. Here's the setup:

var
  Controls: TArray<TControls>;
  Items: ^TArray<TControls>;
begin
  Items := @Controls;
end;

The problem: When Controls is resized (e.g., via SetLength()), Delphi may reallocate the array, invalidating the pointer stored in Items.

1. Does Delphi always reallocate memory when resizing a dynamic array?

2. How can I ensure Items always points to the updated Controls?

3. Is using a pointer the best approach here, or should I use another mechanism?

Answer 1

I think you are a bit confused.

First, recall that Delphi dynamic arrays are reference types (without copy-on-write (COW) semantics). The value of the dynamic array variable is a pointer to a dynamic array heap object.

So, if you have

var
  A: TArray<Integer>;
  B: TArray<Integer>;
begin
  A := [1, 2, 3, 4];
  B := A;

then A and B will both be pointers under the hood, and they will point to the same dynamic array heap object, which has a reference count of 2:

Hence, if you try something like

  Writeln(NativeInt(A));
  Writeln(NativeInt(B));

you'll get the same address (here: 02E62D48).

Now, if you then call SetLength on A, then the RTL may of course need to reallocate and create a brand new heap object at some other address. In fact, this will always happen if the original heap object has a reference count of more than 1. From the documentation:

Following a call to SetLength, [the argument array] S is guaranteed to reference a unique string or array -- that is, a string or array with a reference count of one.

And then B will be left pointing to the old heap object, which now has reference count 1 too.

For example, if I redo my Writeln(NativeInt(A)); Writeln(NativeInt(B)) I may get

A initially:  $02E62D48 (refcount 2)
B initially:  $02E62D48
A after:      $032CB378 (refcount 1)
B after:      $02E62D48 (refcount 1)

Now, your code

var
  A: TArray<Integer>;
  B: ^TArray<Integer>;
begin
  A := [1, 2, 3, 4];
  B := @A;

does something different. Here, B points to the A variable itself, which in turns points to the dynamic array heap object (with a reference count of only 1).

Hence, to go from A to the heap object, you follow one pointer. But to go from B to the heap object, you follow two pointers. First you follow B to find A, and here you find the desired address of the heap object.

Calling SetLength(A) may indeed change the value of A (the address to the dynamic array heap object), but it will never change the address @A of A.

Hence, B will still be valid. It will point to the same address @A, but here there may be a different address, the new address to the dynamic array heap object.

To see this, try, for example,

  A := [1, 2, 3, 4];
  B := @A;

  Writeln(NativeInt(A));
  Writeln(NativeInt(B));

  SetLength(A, 66);

  Writeln(NativeInt(A));
  Writeln(NativeInt(B));

This will write, for example,

43199816 (first address of array heap object)
6531208 (address of local variable A)
43168632 (new address of array heap object)
6531208 (address of local variable A)

But if you access B^ instead, you get the same object (different run):

53161288 (first address of array heap object)
53161288 (first address of array heap object)
53130104 (new address of array heap object)
53130104 (new address of array heap object)

So calling SetLength on A will not ruin your B pointer. Again, this is because B points to the local variable A (which won't move), which in turn contains the always up-to-date address to the array heap object associated with A.

Does Delphi always reallocate memory when resizing a dynamic array?

There is no guarantee that the heap object won't change address. If the heap object initially had a refcount of > 1, then it will get a new address. In general, expect the address to be changed.

How can I ensure Items always points to the updated Controls?

It will always do that.

Is using a pointer the best approach here, or should I use another mechanism?

In modern Delphi, pointers are rarely the "right" approach unless you are an expert developer doing something fairly advanced.

I don't know your context, but maybe you'd be better off with a modern TList<Integer>.

Answer 2

When Controls is resized (e.g., via SetLength), Delphi may reallocate the array, invalidating the pointer stored in Items.

That is incorrect. You are pointing at the Controls variable itself, not at the array which Controls refers to. Resizing the array will not change the variable, and thus will not invalidate your pointer.

Had you instead pointed at the actual array, THEN you would have the problem you described, eg:

type
  PControls = ^TControls;
var
  Controls: TArray<TControls>;
  Items: PControls;
begin
  SetLength(Controls, ...);
  Items := @Controls[0];
  // or:
  // Items := PControls(Controls);
  SetLength(Controls, ...); // <-- Items invalidated here!
  ...
end;

Does Delphi always reallocate memory when resizing a dynamic array?

Unfortunately, yes. The RTL is not smart enough to avoid a reallocation if the new array size is less than or equal to the current array size.

How can I ensure Items always points to the updated Controls?

It already does, in your example.