Backtesting with Sierra Chart (SC) - futures io
futures io



Backtesting with Sierra Chart (SC)


Discussion in Sierra Chart

Updated
      Top Posters
    1. looks_one LittleFinger with 7 posts (0 thanks)
    2. looks_two bbking with 4 posts (1 thanks)
    3. looks_3 Jemo with 1 posts (0 thanks)
    4. looks_4 chris123 with 1 posts (0 thanks)
    1. trending_up 2,624 views
    2. thumb_up 1 thanks given
    3. group 6 followers
    1. forum 12 posts
    2. attach_file 0 attachments




Welcome to futures io: the largest futures trading community on the planet, with well over 125,000 members
  • Genuine reviews from real traders, not fake reviews from stealth vendors
  • Quality education from leading professional traders
  • We are a friendly, helpful, and positive community
  • We do not tolerate rude behavior, trolling, or vendors advertising in posts
  • We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community.  It's free and simple.

-- Big Mike, Site Administrator

(If you already have an account, login at the top of the page)

 
Search this Thread
 

Backtesting with Sierra Chart (SC)

(login for full post details)
  #1 (permalink)
chris123
Frankfurt + Germany
 
 
Posts: 4 since Apr 2018
Thanks: 0 given, 1 received

Hello,

at the moment I backtest my system and try to find an appropriate exit strategy.

I use ACSIL Programming Concepts and have Tick Data available.
After a backtest I check the results using Trade Activity Log, especially columns Profit/Loss, Runup, Drawdown.

I can see that there are sometimes huge price gaps between the price that fulfills my exit condition and the price shown in the trade log. This makes it impossible to test and optimize exit strategies like trailing stops for me.

Example:
I use a trailing stop of 100$ based on the Runup in the code, e.g. stop = runup - $100.
Even if the position is in profit with runups way beyond $100, I see losses in the P&L row after the backtest
that are beyond any tolerance bands acceptable or slippage.

It seems that the price in the log is always the close price of the bar after the exit condition was fulfilled.
Of course if this is a huge bar the price can be way away from my exit condition.


I use Auto Trade System Bar Based Backtest for this because it is relatively fast.

If I use Auto Trade System Replay Backtest it is extremly slow and also the values recorded in Trade Activity Log seem to be wrong, e.g. in Runup column I see huge runups that do not correspond with the charts and are not identical with the runups when I used Bar Based Backtest, e.g. Bar Based Runup = $ 300, Replay Runup = $ 3995
for the same trade.


Another issue I had with Sierra Chart while backtesting was regarding to the position sizing.

I use an algo to determine position size depending on my risk per trade and the following code:

NewOrder.OrderQuantity = Quantity;

Quantity is a variable filled by the algo mentioned above, e.g. 3 contracts.

When I have quantities > 1 I can see partial order fills while backtests in the trade log when my exit conditions are met, e.g. 3 fills at the same time.

Since my code does not use any scaling out methods and since I always exit on the full position size this is very strange (no help from support).

Maybe someone of the forum users can give me an advice if it makes sense to proceed with Sierra Chart
in general for serious trading systems development.


Default section:

sc.StudyDescription = "This study function is a Testenvironment";
sc.AutoLoop = 1;
sc.GraphRegion = 0;
sc.FreeDLL = 1;
sc.AllowMultipleEntriesInSameDirection = false;
sc.MaximumPositionAllowed = 100000;
sc.SupportReversals = false;
sc.SendOrdersToTradeService = false;
sc.AllowOppositeEntryWithOpposingPositionOrOrders = false;
sc.SupportAttachedOrdersForTrading = false;
sc.CancelAllOrdersOnEntriesAndReversals = true;
sc.AllowEntryWithWorkingOrders = false;
sc.CancelAllWorkingOrdersOnExit = false;
sc.AllowOnlyOneTradePerBar = true;
sc.MaintainTradeStatisticsAndTradesData = true;


Statement

if (sc.GetBarHasClosedStatus() == BHCS_BAR_HAS_NOT_CLOSED) return;

is disabled !


Looking forward to any comments or suggestions.

Regards,

Chris

Reply With Quote

Can you help answer these questions
from other members on futures io?
Multichart NET: where can i find esplanation on CustomIn …
MultiCharts
NT8 connectivity by FXI API to unsupported brokers?
NinjaTrader
AMP discontinuing its co-location server? Which futures …
Brokers
How do you put a consolidation into EL?
EasyLanguage Programming
MBoxWave for Sierra Chart for sale 50%
Platforms and Indicators
 
Best Threads (Most Thanked)
in the last 7 days on futures io
How much do you know about Bitcoin?
129 thanks
I finally blew up an account
43 thanks
The Crude Dude Oil Trading System
35 thanks
Spoo-nalysis ES e-mini futures S&P 500
30 thanks
Big Mike in Ecuador
30 thanks
 
(login for full post details)
  #2 (permalink)
 bbking 
Boston, MA USA
 
Experience: Intermediate
Platform: SierraChart
Trading: ES
 
Posts: 7 since Jun 2018
Thanks: 4 given, 3 received

> if (sc.GetBarHasClosedStatus() == BHCS_BAR_HAS_NOT_CLOSED) return;
> is disabled !

Yikes! If I disable this, then I get fills from the previous bar instead of the bar after my indicator fires.

Reply With Quote
 
(login for full post details)
  #3 (permalink)
 LittleFinger 
Denver Colorado/USA
 
Experience: Intermediate
Platform: SierraChart
Broker: AMP
Trading: ES
 
LittleFinger's Avatar
 
Posts: 108 since May 2017
Thanks: 124 given, 127 received



chris123 View Post
I can see that there are sometimes huge price gaps between the price that fulfills my exit condition and the price shown in the trade log. This makes it impossible to test and optimize exit strategies like trailing stops for me.
....
It seems that the price in the log is always the close price of the bar after the exit condition was fulfilled.
Of course if this is a huge bar the price can be way away from my exit condition.
....
I use Auto Trade System Bar Based Backtest for this because it is relatively fast.
Chris

It sounds like you shouldn't use the bar-based backtest if you need trades to execute mid-bar. Not sure about the other issues. I hope you find a solution. I am getting used to backtesting in SC also.

Reply With Quote
 
(login for full post details)
  #4 (permalink)
 LittleFinger 
Denver Colorado/USA
 
Experience: Intermediate
Platform: SierraChart
Broker: AMP
Trading: ES
 
LittleFinger's Avatar
 
Posts: 108 since May 2017
Thanks: 124 given, 127 received

This thread has a lot of views.

I think a lot of people must be looking for general SC backtesting info or something.

If there are any SC vets here that use the platform for automation or have extensive backtesting experience with it I would be interested in talking to you.

Reply With Quote
 
(login for full post details)
  #5 (permalink)
 bbking 
Boston, MA USA
 
Experience: Intermediate
Platform: SierraChart
Trading: ES
 
Posts: 7 since Jun 2018
Thanks: 4 given, 3 received

I have worked with the bar based backtest in Sierra for a couple of weeks and Iíve convinced myself that it can be wildly inaccurate. The tick by tick replay is supposed to be much more accurate, but Iím still trying to figure out how to make it run consistently.

Happy to collaborate on this, LittleFinger.

Anyone with a similar interest is free to message me.


Sent using the futures.io mobile app

Reply With Quote
 
(login for full post details)
  #6 (permalink)
 LittleFinger 
Denver Colorado/USA
 
Experience: Intermediate
Platform: SierraChart
Broker: AMP
Trading: ES
 
LittleFinger's Avatar
 
Posts: 108 since May 2017
Thanks: 124 given, 127 received


bbking View Post
I have worked with the bar based backtest in Sierra for a couple of weeks and Iíve convinced myself that it can be wildly inaccurate. The tick by tick replay is supposed to be much more accurate, but Iím still trying to figure out how to make it run consistently.

Happy to collaborate on this, LittleFinger.

Anyone with a similar interest is free to message me.


Sent using the futures.io mobile app

I'm glad you brought that up. It is unfortunate the way that the bar-based backtest calculates to distort your results to make them look much better.

From what I've seen, if your exit rules can calculate mid bar, then absolutely do not use the bar-based backtest. It will determine the absolute best price to get you out of the trade, even if it was impossible for you to be filled at that price. It will dramatically inflate the results of any backtest that does mid-bar calculations for entry or exit.

However, if you add sc.GetBarHasClosedStatus() == BHCS_BAR_HAS_CLOSED to your exit logic then you can get close results on the bar based backtest to what you get on a replay test, but there will be some imperfections.

Reply With Quote
 
(login for full post details)
  #7 (permalink)
 bbking 
Boston, MA USA
 
Experience: Intermediate
Platform: SierraChart
Trading: ES
 
Posts: 7 since Jun 2018
Thanks: 4 given, 3 received


LittleFinger View Post
if you add sc.GetBarHasClosedStatus() == BHCS_BAR_HAS_CLOSED to your exit logic then you can get close results

This is an important point to reiterate for anyone else. Setting sc.GetBarHasClosedStatus() == BHCS_BAR_HAS_CLOSED was absolutely critical to prevent the awarding of prices from the before the open point of the bar.

Even so, I found that the "logic" (which is documented) is not really what I want, from the bar based testing. I asked Sierra to give users the ability to control how fills are made, but they have so far studiously avoided even responding to the request.

If you or anyone else would like to be able to control exactly how fills are awarded in your bar backtest, please add a comment in support here:

https://www.sierrachart.com/SupportBoard.php?ThreadID=41741

Reply With Quote
 
(login for full post details)
  #8 (permalink)
 LittleFinger 
Denver Colorado/USA
 
Experience: Intermediate
Platform: SierraChart
Broker: AMP
Trading: ES
 
LittleFinger's Avatar
 
Posts: 108 since May 2017
Thanks: 124 given, 127 received

I agree that some options on the bar based logic would be nice.


I'm currently trying to figure out the best method to limit a strategy to one trade a day.


I was thinking compare current day with day of last fill, but not sure how to get the current day on the chart for backtesting and not actual current day.

Reply With Quote
 
(login for full post details)
  #9 (permalink)
 bbking 
Boston, MA USA
 
Experience: Intermediate
Platform: SierraChart
Trading: ES
 
Posts: 7 since Jun 2018
Thanks: 4 given, 3 received


LittleFinger View Post
I'm currently trying to figure out the best method to limit a strategy to one trade a day.

I was thinking compare current day with day of last fill, but not sure how to get the current day on the chart for backtesting and not actual current day.

a) How to get the start time of the most recent bar, in an ASCIL study; so here I log the bar time:
 
Code
    // define a macro to make it easy to do very verbose logging.
   #define vv(args...) \
       do {SCString Msg; Msg.Format(args); sc.AddMessageToLog(Msg, 0);} while(0);
    int BarStartTime = sc.BaseDateTimeIn.TimeAt(sc.Index);
    int minuteOfDay = BarStartTime / 60;
    int sec = BarStartTime % 60;
    double close = sc.Close[sc.Index];
    vv("at simtm %02i:%02i:%02i at sc.Index %i, close=%f",  
              minuteOfDay / 60, minuteOfDay % 60, sec, sc.Index, close)
b) To process each bar once (advice from the Sierra documentation); could be generalized...

 
Code
   
    if(sc.GetBarHasClosedStatus(sc.Index) != BHCS_BAR_HAS_CLOSED) {
      return;
    }

    //This demonstrates a simple method to prevent processing on a bar more than once.
    //All bars in the chart other than the last one are only going to have processing for\
 them in a study function, done only once.
    //However, the last bar in the chart could be multiple times during real-time updatin\
g and during a chart replay.
    //
    // must come *after* the BHCS_BAR_HAS_CLOSED check!
    //
    int &LastBarIndexProcessed = sc.GetPersistentInt(1);
    if (sc.Index == 0) {
      LastBarIndexProcessed = -1;
    }
    if(sc.Index == LastBarIndexProcessed) {
      return;
    }
    LastBarIndexProcessed = sc.Index;

    // Here, I think if you now reference a static (file scope or global) variable that holds the
    // time of the last trade, you should be able to limit the trading to once per day.

Reply With Quote
The following user says Thank You to bbking for this post:
 
(login for full post details)
  #10 (permalink)
 LittleFinger 
Denver Colorado/USA
 
Experience: Intermediate
Platform: SierraChart
Broker: AMP
Trading: ES
 
LittleFinger's Avatar
 
Posts: 108 since May 2017
Thanks: 124 given, 127 received


I'm getting my feet wet still with programming so I'm going to have to work through your example slowly.

Thanks for that

Reply With Quote
 
(login for full post details)
  #11 (permalink)
 LittleFinger 
Denver Colorado/USA
 
Experience: Intermediate
Platform: SierraChart
Broker: AMP
Trading: ES
 
LittleFinger's Avatar
 
Posts: 108 since May 2017
Thanks: 124 given, 127 received

So I put this on hold for a bit because I was working on some other things, but by using some nifty ACSIL features I was able to compare the current day with the day that the last fill occurred on.

You need to get date values: https://www.sierrachart.com/index.php?page=doc/SCDateTime.html

// generate position data:
s_SCPositionData PositionData;
sc.GetTradePosition(PositionData) ;

// define variable for the current date
int CurrentDay = sc.BaseDateTimeIn[sc.Index].GetDate();

// define variable for the last fill date
int LastFillDay = PositionData.LastFillDateTime.GetDate();

// make this a condition of your trade entry logic if you want to limit the strategy to one trade a day
CurrentDay > LastFillDay



Reply With Quote
 
(login for full post details)
  #12 (permalink)
 Jemo 
FLorida / USA
 
Experience: Intermediate
Platform: TOS, TS, Sierra Charts
Broker: TS, MC,NT, TOS
Trading: Index, Commodities, Currencies
 
Posts: 21 since Apr 2017
Thanks: 30 given, 14 received

I am very new to backtesting and I was wondering if you can recommend a book/site/source where I can start learning the fundamentals of backtesting, preferably using sierra charts.

All I could find after doing a search on support of of SC, site that sells a backtesting software and no info on how to use SC for this purpose (profsoftware.com)

thank you

Follow me on Twitter Reply With Quote
 
(login for full post details)
  #13 (permalink)
 LittleFinger 
Denver Colorado/USA
 
Experience: Intermediate
Platform: SierraChart
Broker: AMP
Trading: ES
 
LittleFinger's Avatar
 
Posts: 108 since May 2017
Thanks: 124 given, 127 received

The SC site has all the info for backtesting with their software. Everything I know about SC I learned from the documentation they provide and asking the occasional question on their support forum as well as searching through the posts for things already answered. I would recommend using google to search the SC site, because it works a little better than the site's built in search. Ex: type into google search: "site: sierrachart.com backtesting". Before you start coding and getting fancy, you can simply replay the chart and make simulated trades to test your ideas. As far as using good methods to gather accurate statistics, Kevin Davey has good recommendations and has a free Monte Carlo simulation spreadsheet that you can download from his site. He has many webinars you can watch on here.

Also they have a spreadsheet system that allows you to write automated strategies using the calculation format, cell references, and formulas that work in excel. So if you already can do basic excel stuff that is a good way to start. It is not as efficient or capable as using the C++ based ACSIL, but it's really fun to play with to test out some basic ideas and have an automated test run with a statistics report generated afterwards.

Another thing I do sometimes is load the spreadsheet study and throw whatever indicators are interesting to me on the chart. Then set the spreadsheet study to use a whole lot of rows so that it populates the spreadsheet with the values of the indicators as far back as you want. If you have your chart set to 5m bars then you will have readings for all of your indicators for every 5 minutes as far back as it goes with the amount of rows that you populate. You can then copy all of that data from the SC spreadsheet inside trhe program and paste it to an actual excel spreadheet to do analysis and calculation on.

Example:
Say you want to know the average distance of price (in ticks) from the 200 period moving average of a 1 hour chart on Fridays between 1-2pm. You get all the price and moving average readings from the SC spreadsheet and in excel you simply filter for the day and time period you want. Find the difference for each row, then get the average. Create an average for when the price is below the 200 and an average for when it is above by splittign up the positive and negative values. before averging. Now you have new information and can start building a database of things like this for the instruments you trade. It gets very tedious at a certain point when you have a lot of data, but you can take this pretty far and excel can do a lot. There are free statistical add-on packages you can use in excel that are great for strategy development.

This is very basic for long time excel users so if I am over-explaining I apologize. I find it to be extremely useful and some never give it a chance so I thought I'd share

Reply With Quote


futures io Trading Community Platforms and Indicators Sierra Chart > Backtesting with Sierra Chart (SC)


Last Updated on September 30, 2019


Upcoming Webinars and Events
 

NinjaTrader Indicator Challenge!

Ongoing
 

Our 12-year anniversary w/ $$,$$$ prizes (check soon)

June
     



Copyright © 2021 by futures io, s.a., Av Ricardo J. Alfaro, Century Tower, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada), info@futures.io
All information is for educational use only and is not investment advice.
There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
no new posts