Memory limits exceeded

The most common cause of this error is the retrieval of custom objects and collections from request.*() functions such as request.security(). Other possible causes include unnecessary drawing updates, excess historical buffer capacity, or inefficient use of max_bars_back().

The sections below explain these issues and potential solutions to fix them.

Requesting collections and other objects with ​request.*()​ calls

The “Memory limits exceeded” error most often occurs when a script calls request.security() or other request.*() functions to create custom objects or collections on the dataset for another symbol or timeframe.

When a script executes a data request, the retrieved data is copied and stored in memory on each bar, enabling the script to reference that data on subsequent bars as it executes across the main dataset. Storing requested results for each bar can consume a lot of memory, depending on the type of data. Requesting the references (IDs) of large collections on each bar can easily lead to excessive memory consumption.

For example, the script below calls request.security() to request the result of a user-defined function call evaluated on the “1D” timeframe. The custom function (dataFunction()) creates a “float” array and assigns its ID to a persistent variable declared using the `var` keyword. On each bar, the function pushes a new balance of power (BOP) value into its array and returns the array’s ID. Each time that the request.security() call evaluates the dataFunction() call, it creates a new copy of the array and returns that copy’s ID. Storing a new copy of the array on each bar requires a lot of memory. Consequently, this script can exceed the memory limits when it runs on datasets that have a lengthy history:

How do I fix this?

Optimize the script’s data requests and limit the data returned by the requests to ensure that only the minimum required data is preserved in memory.

The typical solution to minimize the saved data from a request depends on the script’s requirements:

• If the script must access only the latest state of a requested collection or other object, structure the request to return that object’s ID on the last bar only.
• If the script must access a requested object in the main context only when specific conditions occur, structure the request to not return the object’s ID when it is not required.
• If the script performs specific calculations on data from a requested collection or other object, and it does not require access to that object in other parts of the main context, structure the request to perform those calculations internally and return the result rather than returning the object’s ID.

The sections below explore these scenarios using the initial example above.

Return the last state only

If a script requires only the latest state of a requested collection, use a conditional structure or expression with barstate.islast as the condition to limit retrieving a copy of that collection to the last bar where the latest state is available.

For example, suppose we want our initial script to plot the BOP value and display the current average in a table. Users cannot see the past results displayed by a table as the script loads on historical bars. Only the final result as of the latest bar is visible to users. Therefore, to minimize the script’s memory demands, we can structure the script’s request to return a usable ID on the last bar only.

Below, we modified the script’s custom function to return a two-item tuple. On the last bar, the function returns the BOP value and the array’s ID. On all previous bars, it returns only the BOP value and an na array ID. The script requests the result of a call to this revised function. It plots the retrieved BOP value on each bar, then calculates and displays the requested array’s average in a table on the last bar if the retrieved array ID is not na:

Return IDs on some bars

If a script needs to access a requested collection or other object in its main context, but not on every bar, structure the request to use conditional structures or expressions that return the object’s ID on only necessary bars, and na on other bars. The logic in the script’s main context can then handle the na gaps in the requested series as necessary.

Suppose we want to modify our initial script to calculate the average, high, and low daily BOP for each completed month. We can structure the script’s request to collect BOP values across the days in each month and return an array ID for the necessary calculations only after a month ends. The dataFunction() function in the following script version pushes BOP values for each bar in a month into an array and returns a two-item tuple. The tuple contains the BOP value and the array’s ID only on the first bar of a month. On other bars, the tuple contains the BOP value and an na ID. The script uses a call to this updated function in its data request. On bars where the requested array ID is not na, the script calculates the average, high, and low values from that array, then assigns the results to persistent variables for plotting:

Return calculated results

If a script requires the result of a calculation involving a collection or other large object, and the script does not need to access that object in its main context, define a function that performs the calculation and returns the result directly rather than returning the underlying object’s ID. Then, use a call to that function as the expression argument in the request.*() call. Avoiding returning object references from requested contexts altogether when they are unnecessary helps minimize memory demands.

For example, we can directly resolve the error in our initial script by structuring the custom dataFunction() function to perform the average calculation and return the result without returning an array ID on any bar. The function in the script version below returns a tuple containing only the BOP and average value. The script requests the result of a call to the updated function and directly plots the calculated values. With this change, the request does not cause any copies of the dataFunction() call’s internal array to persist in memory in the main context:

Other possible error sources and their fixes

There are a few other factors that can significantly impact a script’s memory demands, including:

• An excessive use of request.*() function calls.
• An unnecessary or improper use of the max_bars_back parameter.
• Excessive historical buffer calculations.
• Inefficient drawing management.
• The generation of many strategy orders and trades.

The sections below explain the best practices for reducing these issues to optimize a script’s memory use.

Minimize ​request.*()​ calls

The request.*() functions can be computationally expensive to call, because they retrieve data from additional datasets and perform extra calculations on that data. Data requests often require significant runtime and memory resources. Excessive or inefficient data requests can easily cause a script to reach its memory limit.

This memory consumption is especially substantial for scripts that request data from lower timeframes with request.security_lower_tf(), because the function returns the IDs of arrays that contain intrabar data for each bar in the script’s main dataset. For example, a script that requests one-minute data from a “1D” chart creates a new array containing hundreds of intrabar data points on each bar. The system must allocate sufficient memory to store each of those large arrays so that the script can access them later in the main context. Maintaining that much data in memory requires a significant amount of resources.

Programmers can reduce the memory requirements of a script’s requests by:

• Removing unnecessary request.*() function calls from the script.
• Changing the timeframe of a request to a higher timeframe to reduce the number of retrieved data points.
• Combining multiple requests to the same dataset into a single request.*() call.
• Using the request.*() function’s calc_bars_count parameter to limit the number of historical bars in the requested dataset.

See the Minimizing request.*() calls section of the Profiling and optimization page to learn more about optimizing data requests.

Use ​max_bars_back​ only when necessary

The max_bars_back parameter of an indicator or strategy script sets the size of the historical buffers for all series in a script. Each buffer determines the number of historical data points that the system maintains in memory for the script’s variables and expressions.

By default, the Pine Script^® runtime system attempts to automatically define an appropriate buffer for each variable and expression. Therefore, using the max_bars_back parameter or max_bars_back() function is typically necessary only in the rare case where the system cannot determine a sufficient buffer size. See the error page The requested historical offset (X) is beyond the historical buffer’s limit (Y) to learn about the potential causes of this issue.

If you encounter a buffer error and must manually set the sizes of a script’s historical buffers by using the max_bars_back parameter or the max_bars_back() function, choose the smallest buffer sizes that accommodate your script’s historical references. Setting buffer sizes to store more data than necessary causes an excessive use of memory resources.

Minimize historical buffer calculations

As explained above, the Pine Script runtime system creates historical buffers for all variables and expressions in a script. It determines the size of each buffer based on the historical references that the script performs via the [] history-referencing operator or the functions that reference history internally.

As a script loads on a dataset, historical references to distant points in the dataset can cause the system to reload the script and increase the size of necessary historical buffers. Each increase to historical buffer sizes leads to increased memory consumption. In some cases, excessive buffer resizing can cause the script to exceed the memory limits. Therefore, ensure a script references only necessary historical data in its calculations. When possible, modify the script’s logic to avoid referencing very distant points in history.

Specifying a calc_bars_count argument in the indicator() or strategy() declaration statement can help reduce memory issues, because it restricts the number of historical bars that the script can use for its calculations. Similarly, using the max_bars_back() function to manually define the appropriate size for a buffer can help reduce buffer calculations. When using this function to specify the size of historical buffers, choose the smallest possible size that accommodates the script’s historical references, to avoid unnecessary memory use.

To learn more about historical buffer calculations and how to optimize them, see the Minimizing historical buffer calculations section of the Profiling and optimization page.

Reduce drawing updates for tables

Tables display only their latest state as of the last available bar on the chart. The previous states of a table on historical bars are not visible to script users. Therefore, updating and displaying tables across historical bars is typically unnecessary. To use the least memory in table drawings, prevent unneccessary updates by defining logic that creates the table object once and then populates it on only the last bar.

To create a table object on only one bar, assign the result of the table.new() call to a variable declared with the var keyword. Then, populate the table on the last bar by placing calls to table.cell() or the table.cell_set_*() functions inside a conditional structure that uses barstate.islast as its condition. See the Tables page to learn more.

Do not update drawings on historical bars

As with tables, all other drawing objects, such as lines and labels, show only their latest states after the script loads on the chart. Users cannot see the visuals update as the script executes across history; users see only the drawing updates that occur on realtime bars.

To help reduce a script’s runtime and memory use, eliminate updates to drawings on historical bars wherever possible. Refer to the Reducing drawing updates section of the Profiling and optimization page for more information.

Minimize the total drawings stored for a chart

Drawing objects, such as lines and labels, can consume a lot of memory, especially if a script recreates drawings unnecessarily.

For example, if a script draws a line between two points, and then needs to update the line’s coordinates later, newcomers to Pine might opt to delete the existing line and create a new one with the updated coordinates. However, such an approach is inefficient and leads to unnecessary resource consumption. The more efficient approach is to use the built-in setter functions for the drawing type, such as line.set_x2(), to modify the coordinates of the original drawing without creating a new object.

In general, to help minimize the resource usage of drawings, optimize them by using any of the following approaches:

• Eliminate unnecessary redrawing by assigning a drawing object’s ID to a variable declared with the var keyword and modifying that object with the available setter functions.
• Remove unnecessary drawings with the *.delete() functions (e.g., line.delete() and label.delete()) to release resources.
• Reduce a script’s drawing limits by specifying arguments for the max_lines_count, max_labels_count, max_boxes_count, or max_polylines_count parameters of the indicator() or strategy() declaration statement.

Filter dates in strategies

The total number of trades or orders simulated by strategies can impact memory consumption. When running strategy scripts that generate frequent orders on large datasets, reduce the number of unnecessary historical orders and trades that persist in memory by limiting the starting point of the strategy.

To limit the starting point of a strategy, a simple and effective approach is to use a conditional structure that activates the strategy’s order placement commands only when the bar’s opening or closing time comes after a specified date.

See the How do I filter trades by a date or time range? portion of our Strategies FAQ page for an example of this technique.