Coupling
Just a quick mention but naturally this proposed solution couples and fuses your element with its central container (or if "container" seems too high-level for your needs, you can just think of it as a "memory pool"), making the two indivisible. But I'd say it's a practical thing to do that has a lot of benefits at little practical cost.
Because what sort of practical flexibility is there to, say, decouple a pixel from the image that owns it, and allows pixels to be allocated and destroyed individually? And similarly for particles of a particle system? Because I see little, if any, but there is almost certainly a tremendous cost to such decoupling in this particular context.
So I'd say don't worry about it. If your design conceptually calls for some central container, or at least has a strong enough performance-critical need that it could benefit from an organized memory allocation/pooling strategy, then tailor your design around this idea of "insertion/removal" over "creation/destruction" or "allocating/freeing", and reap the benefits -- of which first and foremost is the elimination of the need to worry about your designs encouraging too many malloc/free calls.