The Wayback Machine - https://web.archive.org/web/20200609091628/https://github.com/quantopian/zipline/issues/2635
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

limit buy orders with daily data not working as expected #2635

Open
quantSystemDev opened this issue Jan 27, 2020 · 0 comments
Open

limit buy orders with daily data not working as expected #2635

quantSystemDev opened this issue Jan 27, 2020 · 0 comments

Comments

@quantSystemDev
Copy link

@quantSystemDev quantSystemDev commented Jan 27, 2020

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

  • Operating System: Windows 10 Home, 64-bit
  • Python Version: 3.5
  • Python Bitness: $ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))'
  • How did you install Zipline: conda
  • Python packages:
    _nb_ext_conf 0.4.0 py35_1
    alabaster 0.7.10 py35_0
    alembic 0.7.7 py35_0 Quantopian
    anaconda-client 1.6.3 py35_0
    astroid 1.5.3 py35_0
    babel 2.5.0 py35_0
    bcolz 0.12.1 np111py35_0 Quantopian
    blas 1.0 mkl
    bleach 1.5.0 py35_0
    blosc 1.14.4 he51fdeb_0
    bottleneck 1.2.1 py35h452e1ab_1
    bzip2 1.0.6 vc14_3 [vc14]
    certifi 2016.2.28 py35_0
    chardet 3.0.4 py35_0
    click 6.7 py35_0
    clyent 1.2.2 py35_0
    colorama 0.3.9 py35_0
    contextlib2 0.5.5 py35_0
    cycler 0.10.0 py35_0
    cyordereddict 0.2.2 py35_0 Quantopian
    cython 0.26 py35_0
    decorator 4.1.2 py35_0
    docutils 0.14 py35_0
    empyrical 0.5.0 py35_0 Quantopian
    entrypoints 0.2.3 py35_0
    freetds 1.00.9 vc14_0 [vc14]
    hdf5 1.10.2 hac2f561_1
    html5lib 0.9999999 py35_0
    icc_rt 2019.0.0 h0cc432a_1
    icu 57.1 vc14_0 [vc14]
    imagesize 0.7.1 py35_0
    intel-openmp 2019.4 245
    intervaltree 2.1.0 py35_0 Quantopian
    ipykernel 4.6.1 py35_0
    ipython 6.1.0 py35_0
    ipython_genutils 0.2.0 py35_0
    ipywidgets 6.0.0 py35_0
    isort 4.2.15 py35_0
    jedi 0.10.2 py35_2
    jinja2 2.9.6 py35_0
    joblib 0.14.1
    jpeg 9b vc14_0 [vc14]
    jsonschema 2.6.0 py35_0
    jupyter_client 5.1.0 py35_0
    jupyter_core 4.3.0 py35_0
    lazy-object-proxy 1.3.1 py35_0
    libpng 1.6.30 vc14_1 [vc14]
    logbook 0.12.5 py35_0 Quantopian
    lru-dict 1.1.4 py35_0 Quantopian
    lzo 2.10 h6df0209_2
    mako 1.0.6 py35_0
    markupsafe 1.0 py35_0
    matplotlib 2.0.2 np111py35_0
    mistune 0.7.4 py35_0
    mkl 2018.0.3 1
    mkl_fft 1.0.6 py35hdbbee80_0
    mkl_random 1.0.1 py35h77b88f5_1
    multipledispatch 0.4.9 py35_0
    mysql-connector-python 2.0.4 py35_0
    nb_anacondacloud 1.4.0 py35_0
    nb_conda 2.2.0 py35_0
    nb_conda_kernels 2.1.0 py35_0
    nbconvert 5.2.1 py35_0
    nbformat 4.4.0 py35_0
    nbpresent 3.0.2 py35_0
    networkx 1.11 py35_0
    notebook 5.0.0 py35_0
    numexpr 2.6.8 py35h9ef55f4_0
    numpy 1.16.1
    numpy 1.11.3 py35h4a99626_4
    numpy-base 1.14.6 py35h8128ebf_4
    numpydoc 0.7.0 py35_0
    openssl 1.0.2l vc14_0 [vc14]
    pandas 0.18.1 np111py35_0
    pandas-datareader 0.5.0 py35_0
    pandocfilters 1.4.2 py35_0
    path.py 10.3.1 py35_0
    patsy 0.4.1 py35_0
    pickleshare 0.7.4 py35_0
    pip 9.0.1 py35_1
    pip 20.0.1
    prompt_toolkit 1.0.15 py35_0
    psutil 5.2.2 py35_0
    pycodestyle 2.3.1 py35_0
    pyflakes 1.6.0 py35_0
    pyfolio 0.9.2
    pygments 2.2.0 py35_0
    pylint 1.7.2 py35_0
    pymssql 2.1.3 py35_0
    pymysql 0.7.9 py35_0
    pyparsing 2.2.0 py35_0
    pyqt 5.6.0 py35_2
    pytables 3.4.4 py35he6f6034_0
    python 3.5.4 0
    python-dateutil 2.6.1 py35_0
    pytz 2017.2 py35_0
    pyyaml 3.12 py35_0
    pyzmq 16.0.2 py35_0
    qt 5.6.2 vc14_6 [vc14]
    qtawesome 0.4.4 py35_0
    qtconsole 4.3.1 py35_0
    qtpy 1.3.1 py35_0
    requests 2.14.2 py35_0
    requests-file 1.4.1 py35_0
    requests-ftp 0.3.1 py35_0
    rope 0.9.4 py35_1
    scikit-learn 0.22.1
    scipy 1.1.0 py35hc28095f_0
    seaborn 0.9.0
    setuptools 36.4.0 py35_1
    simplegeneric 0.8.1 py35_1
    singledispatch 3.4.0.3 py35_0
    sip 4.18 py35_0
    six 1.10.0 py35_1
    snappy 1.1.7 h777316e_3
    snowballstemmer 1.2.1 py35_0
    sortedcontainers 1.5.7 py35_0
    sphinx 1.6.3 py35_0
    sphinxcontrib 1.0 py35_0
    sphinxcontrib-websupport 1.0.1 py35_0
    spyder 3.2.3 py35_0
    sqlalchemy 1.1.13 py35_0
    statsmodels 0.9.0 py35h452e1ab_0
    TA-Lib 0.4.17
    tbb 2018.0.5 he980bc4_0
    tbb4py 2018.0.5 py35he980bc4_0
    testpath 0.3.1 py35_0
    tk 8.5.18 vc14_0 [vc14]
    toolz 0.8.2 py35_0
    tornado 4.5.2 py35_0
    tqdm 4.15.0 py35_0
    trading-calendars 1.11.1 py35_0 Quantopian
    traitlets 4.3.2 py35_0
    vc 14 0
    vs2015_runtime 14.0.25420 0
    wcwidth 0.1.7 py35_0
    wheel 0.29.0 py35_0
    widgetsnbextension 3.0.2 py35_0
    win_unicode_console 0.5 py35_0
    wincertstore 0.2 py35_0
    wrapt 1.10.11 py35_0
    zipline 1.3.0 np111py35_0 Quantopian
    zlib 1.2.11 vc14_0 [vc14]

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

  • What did you expect to happen?

Using daily frequency data, I expected a limit buy order to fill on the current bar at the limit price if the limit price is above the low price - or possibly at the open price if the open price gap is below the limit price.

  • What happened instead?

The only condition for which a limit buy order gets filled seems to be if the limit price is greater than the close price of the next bar. If that condition occurs then the buy order is filled on the next bar at the close price of the next bar. If that condition does not occur, the limit buy order does not get filled even if limit price falls within the open-low-close prices of the current bar (or of the next bar).

Here is how you can reproduce this issue on your machine:

Reproduction Steps

  1. Create a bundle with daily data for some symbol, such as SPY.
  2. Create and run some test code in a notebook similar to the code below.

`
from zipline import run_algorithm
from zipline.api import order_target_percent, symbol, schedule_function, date_rules, time_rules
from zipline.finance.execution import MarketOrder, LimitOrder
from zipline.finance.slippage import FixedSlippage, NoSlippage, SlippageModel
from datetime import datetime
import pytz
import pyfolio as pf
import numpy as np

class InstantSlippage(SlippageModel):
# See #1844 (comment)
# See also #2364 (comment)
# Instant fill on the current bar at the current bar close or open price is not possible with daily data.
# By default market orders are filled on the next bar at the next bar's close price.
# With a custom slippage model we can't change which bar the order is filled, but we can change the fill price.
# With this model - we can set the fill price to be the prior day close or alternatively the current day open.

def process_order(self, data, order):
    # Use price from previous bar
    price = data.history(order.sid, 'price', 2, '1d')[0]
    
    # Alternative: Use current bar's open, instead of close
    #price = data.current(order.sid, 'open')

    return (price, order.amount)

def initialize(context):

context.set_slippage(NoSlippage())
#context.set_slippage(InstantSlippage())  # this works for market ordeers
#context.set_slippage(LimitBuySlippage()) # this does not work for limit orders
    
# Schedule the trading routine
# Note time_rules seem to be ignored with daily frequency data.
schedule_function(trade, date_rules.every_day(), time_rules.market_close()) 
#schedule_function(trade, date_rules.every_day(), time_rules.market_open()) 

# market orders are executing at next day close + slippage regardless of the time rules!
# Limit orders are filled on the next bar only if the limit price is greater than the close of the next bar!
# See https://github.com/quantopian/zipline/issues/2217#issue-334055304

# Which stock to trade
context.spy = symbol('SPY')

def trade(context, data):

# Trade execution with daily data is problematic
    
# Market orders will fill by default at the close price of the next day (assuming no slippage).
# We can use "InstantSlippage" above to set the price of the fill (if not the day of the fill).
#order_target_percent(context.spy, 1.0, style=MarketOrder())

# Limit orders will fill at the close of the next day only if  limit price > next day close price.
order_target_percent(context.spy, 1.0, style=LimitOrder(limit_price=331.8))  # this order gets filled
#order_target_percent(context.spy, 1.0, style=LimitOrder(limit_price=331.4))  # this order does not get filled

open_price = data.history(context.spy, 'open', 1, '1d').iloc[-1]
high_price = data.history(context.spy, 'high', 1, '1d').iloc[-1]
low_price = data.history(context.spy, 'low', 1, '1d').iloc[-1]
close_price = data.history(context.spy, 'close', 1, '1d').iloc[-1]

print(data.history(context.spy, 'close', 1, '1d').index)
print('open = {}'.format(open_price))
print('high = {}'.format(high_price))
print('low= {}'.format(low_price))
print('close = {}'.format(close_price))
print()

def analyze(context, perf):
# Use PyFolio to generate a performance report
returns, positions, transactions = pf.utils.extract_rets_pos_txn_from_zipline(perf)

start = datetime(2020, 1, 22, tzinfo=pytz.UTC)
end = datetime(2020, 1, 23, tzinfo=pytz.UTC)

result = run_algorithm(
start=start,
end=end,
initialize=initialize,
analyze=analyze,
capital_base=10000,
data_frequency = 'daily',
bundle='rising_assets_data'
)

returns, positions, transactions = pf.utils.extract_rets_pos_txn_from_zipline(result)
transactions.tail()`

What steps have you taken to resolve this already?

I tried to create a custom slippage model for limit buys like the code below, but it did not work.
`class LimitBuySlippage(SlippageModel):

def process_order(self, data, order):
   
    limit_price = order.limit
    #open_price = data.current(order.sid, 'open')
    #low_price = data.current(order.sid, 'low')
    open_price = data.history(order.sid, 'open', 2, '1d')[0]
    low_price = data.history(order.sid, 'low', 2, '1d')[0]
    
    price = np.where(open_price<limit_price,open_price,np.where(low_price<limit_price,limit_price,np.nan))

    return (price, order.amount)`

...

Anything else?

I realize it may not be possible to get instant (same day) limit order fills on the current bar with the existing code base. However if there could be a way to create a custom slippage model (similar to the excellent InstantSlippage model #1844 (comment) for market orders) to achieve execution on the next bar at the resulting limit order fill price (if limit is triggered on the current bar), that would be great!
...

Sincerely,
quantSystemDev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
1 participant
You can’t perform that action at this time.