2

I currently have the following code which goes through each row of a dataframe and assigns the prior row value for a certain cell to the current row of a different cell.

Basically what im doing is finding out what 'yesterdays' value for a certain metric is compared to today. As you would expect this is quite slow (especially since I am working with dataframes that have hundreds of thousands of lines).

for index, row in symbol_df.iterrows():
    if index != 0:
        symbol_df.loc[index, 'yesterday_sma_20'] = symbol_df.loc[index-1]['sma_20']
        symbol_df.loc[index, 'yesterday_roc_20'] = symbol_df.loc[index-1]['roc_20']
        symbol_df.loc[index, 'yesterday_roc_100'] = symbol_df.loc[index-1]['roc_100']
        symbol_df.loc[index, 'yesterday_atr_10'] = symbol_df.loc[index-1]['atr_10']
        symbol_df.loc[index, 'yesterday_vsma_20'] = symbol_df.loc[index-1]['vsma_20']

Is there a way to turn this into a vectorized operation? Or really just any way to speed it up instead of having to iterate through each row individually?

Improve this question
0

1 Answer 1

Reset to default
3

I might be overlooking something, but I think using .shift() should do it.

import pandas as pd

df = pd.read_csv('test.csv')
print df

#         Date    SMA_20    ROC_20
# 0  7/22/2015  0.754889  0.807870
# 1  7/23/2015  0.376448  0.791365
# 2  7/22/2015  0.527232  0.407420
# 3  7/24/2015  0.616281  0.027188
# 4  7/22/2015  0.126556  0.274681
# 5  7/25/2015  0.570008  0.864057
# 6  7/22/2015  0.632057  0.746988
# 7  7/26/2015  0.373405  0.883944
# 8  7/22/2015  0.775591  0.453368
# 9  7/27/2015  0.678638  0.313374

df['y_SMA_20'] = df['SMA_20'].shift()
df['y_ROC_20'] = df['ROC_20'].shift()
print df

#         Date    SMA_20    ROC_20  y_SMA_20  y_ROC_20
# 0  7/22/2015  0.754889  0.807870       NaN       NaN
# 1  7/23/2015  0.376448  0.791365  0.754889  0.807870
# 2  7/22/2015  0.527232  0.407420  0.376448  0.791365
# 3  7/24/2015  0.616281  0.027188  0.527232  0.407420
# 4  7/22/2015  0.126556  0.274681  0.616281  0.027188
# 5  7/25/2015  0.570008  0.864057  0.126556  0.274681
# 6  7/22/2015  0.632057  0.746988  0.570008  0.864057
# 7  7/26/2015  0.373405  0.883944  0.632057  0.746988
# 8  7/22/2015  0.775591  0.453368  0.373405  0.883944
# 9  7/27/2015  0.678638  0.313374  0.775591  0.453368
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Everyone starts somewhere. Kindly accept the answer if it helped you. Good luck! (I see you're doing some trading there!)
Sorry, it wouldn't allow me to accept the answer earlier. I think there's a minimum time. But yes, I trade full time. Thanks again for the help.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.