Reallocate dynamic array within struct from function

I have read other questions that are somewhat related to this topic on StackOverflow (namely (1) and (2)), but they do not address my question regarding a member array specified within a struct in a manner that I could follow logically.

In C, I specify a double pointer as a member of my struct. This double pointer purposes as a 2-D dynamic array - strictly speaking, it is an array of pointers to an array of pointers to characters in memory. The rest of the members of the struct are meant to keep track of the current and allocated size of this 2-D dynamic array.

typedef struct CharArrayXY
{
    char **array; // A look-up table pointing to a 2-D arrangement of values stored in memory
    size_t used_x; // Number of arrays occupying 2-D array
    size_t *used_y; // Number of elements occupying each array
    size_t size_x; // Number of spaces allocated for arrays in 2-D array
    size_t *size_y; // Number of spaces allocated for elements in each array
} CharArrayXY;

I initialize all of the members of the struct with several malloc() calls in a separate function without any problems. This is the important bit from that function:

a->array = malloc(sizeof(*a->array) * x_dim);
// Check if it's NULL...
for (x_ind = 0; x_ind < x_dim; x_ind++)
{
    a->array[x_ind] = malloc(sizeof(char) * y_dim);
}

a->size_x = x_dim;

a->size_y = malloc(sizeof(size_t) * x_dim);
// Check if it's NULL...
for (x_ind = 0; x_ind < x_dim; x_ind++)
{
    a->size_y[x_ind] = y_dim;
}

a->used_x = 0;

a->used_y = malloc(sizeof(size_t) * x_dim);
// Check if it's NULL...
for (x_ind = 0; x_ind < x_dim; x_ind++)
{
    a->used_y[x_ind] = 0;
}

I resize the struct's double pointer member with a realloc() call.

void add_char_xy (CharArrayXY *a, char element, size_t x_ind)
{
    if (a->used_x == a->size_x)
    {
        printf("Reallocating for more space...\n");
        a->size_x += 2;
        temp_ptr = realloc(a->array, sizeof(*a->array) * a->size_x);
        if (temp_ptr == NULL)
        {
            fprintf(stderr, "realloc() call failed.\n");
            return;
        }
        a->array = temp_ptr;
    }

    if (a->used_y[x_ind] == a->size_y[x_ind])
    {
        a->size_y[x_ind] += 10;
        a->array[x_ind] = realloc(a->array[x_ind], sizeof(*a->array[x_ind]) * a->size_y[x_ind]);
    }

    printf("Storing '%c' in CharArrayXY[%zu][%zu].\n", element, x_ind, a->used_y[x_ind]);
    a->array[x_ind][a->used_y[x_ind]] = element;
    a->used_y[x_ind]++;
}

The realloc() call does not result in a NULL pointer, which suggests that there is no problem with reallocating the memory. I have traced the source of the problem to the used_y member of the struct, since it is updated to 56 instead of 0. My understanding is that the used_y member was reallocated to a new space in memory that I have not allocated for it, so it returns a trash value when I attempt to access it.

... Storing many characters in the array
Storing 'b' in CharArrayXY[4][0]. <-- Array is being populated fine (hasn't been reallocated yet, still filling initialized array)
Storing 'l' in CharArrayXY[4][1].
Storing 'd' in CharArrayXY[4][2].
Storing 'b' in CharArrayXY[4][3].
Storing 'e' in CharArrayXY[4][4].
Storing 'f' in CharArrayXY[4][5].
Storing 's' in CharArrayXY[4][6].
Storing 'a' in CharArrayXY[4][7].
Storing 'r' in CharArrayXY[4][8].
Storing 'c' in CharArrayXY[4][9].
Reallocating for more space...
Storing 'b' in CharArrayXY[5][65]. <-- Right after the realloc() call
Segmentation fault (core dumped)

Why does it appear that the value of a->used_y[x_ind] is filled with a trash value from memory during after I successfully reallocate the array member of the struct?