NexusFi: Find Your Edge


Home Menu

 





Coding a Journal indicator for SierraChart


Discussion in Sierra Chart

Updated
      Top Posters
    1. looks_one shokunin with 17 posts (53 thanks)
    2. looks_two cory with 1 posts (0 thanks)
    3. looks_3 ninjus with 1 posts (1 thanks)
    4. looks_4 tklee with 1 posts (0 thanks)
    1. trending_up 8,734 views
    2. thumb_up 54 thanks given
    3. group 17 followers
    1. forum 20 posts
    2. attach_file 0 attachments




 
Search this Thread

Coding a Journal indicator for SierraChart

  #11 (permalink)
 shokunin 
Manchester, United Kingdom
 
Experience: Advanced
Platform: Sierra Chart
Broker: Optimus Futures, Rithmic, Denali
Trading: ES
Posts: 85 since Jul 2020
Thanks Given: 6
Thanks Received: 157

Progress

Using hotkeys, I can now:
  • step through orderfills and assign orderfills to opportunities; and
  • display a textbox, to show statistics and record information about the selected opportunity.

Here is a mock-up of the information I want to display and record for each opportunity:


The initial section will be calculated automatically based on my order fills and other information lines I have placed on the chart:
  • Position - the current number of contracts open.
  • P&L ticks - the difference between the average entry price and current price, I tend to think in ticks captured irrespective of position size.
  • Trades - A simple opportunity would have 2 (entry and exit). A strategy that scales-in/scales-out would have more.
  • Total contracts traded - Used to calculate commission and may be used to influence a tilt-o-meter™.
  • Average entry price - Required for strategies that scale-in.
  • Failure price - price if hit would signify the strategy is not running to plan.
  • Target price - initial target that the trade should be expected to meet.
  • Initial risk reward - the assumed risk/reward at the point of entering.
  • MAE - maximum adverse excursion from the average entry price. Also the time spent at a loss.
  • MFE - maximum favourable excursion from the average entry price. Also the time spent in profit.

I also want to record the realistic start and end times of the opportunity. This will let me determine if I am entering trades too late, or taking profits too early.

The remainder of the entries are a self-evaluation based on a 5-star rating:
  • Market structure - how well am I identifying trend lines and narrative.
  • Market conditions - how well am I identifying the current market character and intent.
  • Profit target - did I have a clearly defined profit target and reason, or did I just FOMO in on the trade without thinking?
  • Failure identification - did I have a clear failure pattern with a solid plan to abort, or did I go in with the mindset 'this trade is going to work no matter what'.
  • Strategy selection - did I pick the right strategy and profit/stop targets based on market structure, market conditions, session time remaining?
  • Focus and clarity - am I in the zone or am I lacking energy or affected by some other negative state?
  • Risk acceptance - For example, if I'm up on the day, am I worried about losing prior gains? This could cause me to exit trades early or move to breakeven too soon?
  • Perception of range - Internalisation and awareness of the ATR, and whether the price movement is contracting or expanding (or likely to contract or expand).
  • Confidence in target - What have I seen that gives me confidence in this trade? Am I taking profit too early?
  • Entry and Exit execution - the ability to read order flow, time rotations, and press the right buttons at the right time.
  • Continual risk management - taking a systematic and logical approach to refining stop and profit targets as the trade progresses. Not being blindsided by a bias and over-confidence in the trade, or having invested too much energy into the trade. Sometimes I will let a trade that is nicely in profit turn into a loss as a result of this.

I want to record this information because simply thinking about each helps me. Currently it is a mental checklist, but having it prompted on-screen should help. I want the data entry to be very quick - I'll probably just setup hotkeys 1-5 to record each rating in sequence, so it should take no more than a few seconds. As well as reminding me to consider each of these points, a future plan is to come up with an algorithm for displaying a tilt-o-meter™, plus reminding me to take breaks and keep hydrated, etc.

The next step
I now need figure out how to interface with SierraChart's historical tick data so I can calculate the stats. As I am displaying this information in real time, and I use the same SierraChart instance for live trading, I need to ensure that any longer running calculations do not block SierraChart and yield back within a reasonable time.

Fellow nexusfi.com members have already found a way to access historical trade data, so that is where I'll start.

Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
The space time continuum and the dynamics of a financial …
Emini and Emicro Index
Futures True Range Report
The Elite Circle
Deepmoney LLM
Elite Quantitative GenAI/LLM
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
ZombieSqueeze
Platforms and Indicators
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
61 thanks
Funded Trader platforms
38 thanks
NexusFi site changelog and issues/problem reporting
26 thanks
GFIs1 1 DAX trade per day journal
19 thanks
The Program
18 thanks
  #12 (permalink)
 shokunin 
Manchester, United Kingdom
 
Experience: Advanced
Platform: Sierra Chart
Broker: Optimus Futures, Rithmic, Denali
Trading: ES
Posts: 85 since Jul 2020
Thanks Given: 6
Thanks Received: 157

Getting tick data out of SierraChart and into my own objects

SierraChart presents data in a different format to my own objects. Here are some of the differences:
  • Different types. For example, SC stores timestamps, price and volume as SCDateTime, float, float. My code uses types Timestamp, Price, Size.
  • SC stores Price, Volume and Timestamp for each tick in separate arrays. My code uses a single data struct "Trade" to represent each trade, and each struct is stored in a single contiguous collection.

Here is the psuedocode for a Study I wrote that pulls data from a Trade chart and keeps my objects up to date. SierraChart is single threaded and on a rather old Xeon E5-2697 it takes approximately 60 seconds to transfer one month of tick data from SierraChart to my TimeAndSales object. This includes yielding back to SierraChart after a few ms to ensure SierraCHart continues to operate smoothly. Once the backfill has completed, all new trades are propagated in real time.
 
Code
// Attach this study to a chart set to type: Trade, setting: 1, timezone: UTC.

On Study initialisation:
    Store a pointer to my own TimeAndSales object.
        Use SC's persistent storage mechanism for this.

On Study FullRecalculation:
    Signal to the TimeAndSales object that all data should be reset.

On Hide:
    Temporarily pause sending trade data to the TimeAndSales object.

On Study update:
    Note the current system time.
    Do some basic checks, e.g. ensure the chart is set to UTC timezone.
    Retrieve the TimeAndSales object from persistent storage.
    Retrieve the last bar that was sent.  (On the first ever run, or after a FullRecalculation,
            no bars will have been sent).
    Do:
        Get the timestamp. price, size and other information from SierraChart for the next Bar
                that needs to be sent.
        Convert each into the required format and send to the TimeAndSales object.
        Check the current system time and compare to when we started.  If our runtime allowance
                has been exceeded, exit the loop.
                The loop will continue on the next call to Update.
    Display some text on the chart to show progress, e.g remaining backfill to send, or any errors.
The reason I want to use my own TimeAndSales object is so I can use multiple threads, pass to my algos for trading, or to my machine learning code for backtesting. The data structures are optimised for the hardware I use and I can offload to the GPU for parallel processing.

For this project, my TImeAndSales object will update each Orderfill and Opportunity (using the Observer Pattern) to generate the statistics I require. This will happen in real time so I can monitor my progress while trading.

Started this thread Reply With Quote
Thanked by:
  #13 (permalink)
 shokunin 
Manchester, United Kingdom
 
Experience: Advanced
Platform: Sierra Chart
Broker: Optimus Futures, Rithmic, Denali
Trading: ES
Posts: 85 since Jul 2020
Thanks Given: 6
Thanks Received: 157


Dealing with out-of-sequence bars

I found that a few of the timestamps in the SierraChart arrays are out of sequence. There are two possible reasons for this; either the data sent from the SC historical servers is incorrect, or the time on the local PC has changed while downloading. SierraChart provide an indicator "Out of Order Timestamps Detector" that highlights bars that are out of sequence.

I am writing a custom study that populates my own timeAndSales object. Each time a new bar is received, the study feeds that bar into timeAndSales. timeAndSales then notifies a condition_variable so that my dispatcher can schedule my algos and indicators, which are running on different threads. As my algos and indicators operate in real time and use binary search on historical data for performance, it is necessary that the bars are sent in the correct sequence.

Here is my plan:

SierraChart ASCIL provides a function sc.StartDownloadHistoricalData that will delete and re-download data from a specific point. As my study iterates through each bar, if it detects an out-of-sequence bar, it will call this function to automatically download data from the last in-sequence bar. It will also note the bar for future reference.

If a download doesn't fix the problem, then the problem is with the data on SierraChart's servers and an error will be logged so I can raise a support ticket with SierraChart.

Here is the snippet of code:
 
Code
Timestamp timestamp = SCDateTimeToTimestamp(sc, sc.BaseDateTimeIn[slotId]);

if (slotId == 0) {
    goto recordTrade;
}

checkOutOfSequence:
{
    Timestamp previous = SCDateTimeToTimestamp(sc, sc.BaseDateTimeIn[slotId-1]);
    if (timestamp < previous) {
        std::string msg = "Out of sequence trade detected: " +
                          std::to_string(slotId-1) + "=" +
                          previous.toString() + ", " +
                          std::to_string(slotId) + "=" +
                          timestamp.toString();
        sc.AddMessageToLog(msg.c_str(), 1);

        checkAlreadyReloadedHistorical:
        {
            if (std::find(pd->outOfSequenceSlotIds.begin(),
                          pd->outOfSequenceSlotIds.end(),
                          slotId)
                      != pd->outOfSequenceSlotIds.end()) {
                sc.AddMessageToLog("Refresh already attempted.", 1);
                std::string msg2 = "Using dummy timestamp for slotId " +
                                   std::to_string(slotId) + ": (" +
                                   "was:" + timestamp.toString() + ", " +
                                   "now:" + previous.toString() + ")";
                sc.AddMessageToLog(msg2.c_str(), 1);
                timestamp = previous;
                goto recordTrade;
            }
        }

        reloadHistorical:
        {
            SCString msg3 = "Reloading historical data from ";
            msg3 += sc.DateTimeToString(sc.BaseDateTimeIn[slotId],
                                       FLAG_DT_COMPLETE_DATETIME_MS);
            sc.AddMessageToLog(msg3, 1);
            sc.StartDownloadHistoricalData(sc.BaseDateTimeIn[slotId]);
            pd->outOfSequenceSlotIds.push_back(slotId);
            pd->modify_timeAndSales().Reset();
            return;
        }
    }
}

recordTrade:
{
    Price price = sc.Close[slotId];
    Size size = sc.Volume[slotId];
    pd->modify_timeAndSales().Record(Trade(timestamp, price, size));
}

OMG, I used a goto!

I tend to write code with the following style objectives:
  1. if statements should contain the exception, not the norm. This means the 'exceptional' execution path is indented (to remind me it is exceptional) and the 'normal' execution path is towards the left margin where it is easier to read.
  2. Separate code blocks into functions where there is good reason to do so, e.g. for code reuse and unit-testing.
  3. If there is no good reason to move code to a function right now, leave the code inline. IMO functions make code harder to read as a) parameters need to be passed, b) it can create code-bloat, and c) the function needs to be defined above the call point which means I need to jump around the page and remember the return point.
  4. Write blocks of code so they can easily be turned into a future function. My method of doing this is to use labels. To avoid compiler warnings on unused labels, I use #pragma warning(disable: 4102).
  5. If a goto statement is easier to read than a state variable and a while loop or if block, prefer the goto.


Example 1:
 
Code
void someFunction(Foo *foo) {
    if (foo != nullptr) {
        // do something
    }
}
This is horrible. Normal execution is contained in an if block and indented.


Example 2:
 
Code
void someFunction(Foo *foo) {
    if (foo == nullptr) { return; }
    // do something
}
A guard statement is placed at the top of the function. Once the guard statement is passed, it can be forgotten about. This makes for easier reading.


Example 3:
 
Code
if (slotId == 0) {
    goto recordTrade;
}

// Do the out-of-sequence-check

recordTrade:
// record the trade
This is the style I used in my code snippet above. My reason for using a goto is similar to using a guard in example 2.


Example 4:
 
Code
if (slotId > 0) {
    //Do the out-of-sequence-check-here;
}

// record the trade
I find example 3 is easier to read and debug. Imagine stepping through the code from top to bottom when slotId == 0. In example 3, I can see exactly where the code branches to and my IDE can take me directly to the label. The label is descriptive so the code is also self-documenting. In example 4, I need to manually hunt for the bottom curly brace. To avoid this, the if block in Example 4 is typically moved to a new function, as in the next example.


Example 5:
 
Code
bool OutOfSequenceCheck(lots-of-parameters-including-non-const-refs) {
    //Do the out-of-sequence-check-here;
}

if (slotId > 0) {
    isRefreshRequested = OutOfSequenceCheck(lots-of-parameters-including-non-const-refs);
    if (isRefreshRequested) {
        return;
    }
}

// record the trade
If the only reason to introduce the OutOfSequenceCheck function is to make example 4 more readable, IMO it hasn't done that. It has added code-bloat, introduced non-top-to-bottom flow, an undocumented return bool that needs to be tracked, and a timestamp that needs to be passed by non-const ref. In order for code to be self-documenting and avoid non-const refs, OutOfSequenceCheck would need to return a struct comprising bool refreshRequested and Timestamp timestamp. This adds even more code bloat.

I have heard some companies get paid by line of code...

Started this thread Reply With Quote
  #14 (permalink)
 
cory's Avatar
 cory 
virginia
 
Experience: Intermediate
Platform: ninja
Trading: NQ
Posts: 6,098 since Jun 2009
Thanks Given: 877
Thanks Received: 8,090

1-4 so true!

Reply With Quote
  #15 (permalink)
 shokunin 
Manchester, United Kingdom
 
Experience: Advanced
Platform: Sierra Chart
Broker: Optimus Futures, Rithmic, Denali
Trading: ES
Posts: 85 since Jul 2020
Thanks Given: 6
Thanks Received: 157

Self-documenting code

One of this things I try and achieve is code that is so obvious that documentation isn't required.


Example1 - A function that returns something. However, it isn't clear without reading the function exactly what is returned:
 
Code
bool DoSomeStuff() {
    // do some stuff
    if (something) {
        return true;
    }
    return false;
}


Example 2 - One way is by documentation, but this is clumsy.
 
Code
// Returns true if it has finished doing all stuff, false if there is still stuff remaining to do.
bool DoSomeStuff();
With booleans, I find it difficult to remember if true=isFinished or true=isRemaining, and I find myself needing to refer to the docs often.


Example 3 - Making the code obvious. In this example it should be clear, without documentation, what is being returned.
 
Code
typedef bool IsFinished;

IsFinished DoSomeStuff();


I'll show a use case for this, especially important with SierraChart, in the next post.

Started this thread Reply With Quote
Thanked by:
  #16 (permalink)
 shokunin 
Manchester, United Kingdom
 
Experience: Advanced
Platform: Sierra Chart
Broker: Optimus Futures, Rithmic, Denali
Trading: ES
Posts: 85 since Jul 2020
Thanks Given: 6
Thanks Received: 157

Keeping SierraChart Responsive

SierraChart calls each custom study every time something happens. E.g. a new trade is received, an order is filled, the mouse is clicked, a drawing is moved, a key is pressed, a related study updates.

The important thing to note is that while the custom study is running, it is blocking SierraChart from doing anything else. Menus will be unresponsive, orders won't process and charts won't update. In some cases, Windows Task Manager will report SierraChart as Not Responding.

Therefore, each study needs to execute very quickly and return control back to SierraChart as soon as possible.

Here are the 2 methods I use to ensure a responsive system:
  1. Yield after a specific time has passed and defer remaining work to the next call.
  2. Put the work onto a queue and have another thread do the work.

For one-off jobs such as loading TimeAndSales data into my own objects, I would generally use method 1. Here is the pseudo code:
 
Code
Make a note of the time.
Recall from Sierra Chart's persistent storage how far we got to last time.
While (time hasn't exceeded our maximum runtime allowance)
{
  // Do work
}
Store in Sierra Chart's persistent storage how far we got to this time.
Return
This type of design pattern is called a Generator or a Co-Routine.

It is possible SierraChart generates new data faster than my study can process in the allotted time. This results in an ever-growing backlog which is unsuitable for a real-time environment like trading.

To avoid this, most of my algorithms run on separate threads. When SierraChart calls my study, the study simply places the update onto a queue. A dispatcher watches the queue, and distributes the update to each algorithm. Because the algorithms can do the work in parallel and can run continuously, this is much more efficient and there is unlikely to be a backlog due to a sudden high number of trades.

In the case of my journaling software, I have an algorithm to calculate the MAE, MFE, Time spend in profit/loss, and all the other statistics I require for each Session, Opportunity and Orderfill.

The SierraChart Journal study I am writing polls the algorithms and displays their status on the chart.

Started this thread Reply With Quote
Thanked by:
  #17 (permalink)
 shokunin 
Manchester, United Kingdom
 
Experience: Advanced
Platform: Sierra Chart
Broker: Optimus Futures, Rithmic, Denali
Trading: ES
Posts: 85 since Jul 2020
Thanks Given: 6
Thanks Received: 157

Psychology

I'm trying to make trading "more like a video game". The reason is to shift focus away from P&L and towards good trading. I want my dopamine hits to come from good stop discipline, favourable risk/reward, sensible risk management, clear identification of market structure and perfect execution.

SierraChart has UTF-8 support, so I've been looking at emojis as a way to indicate an opportunity outcome.

0xf0 0x9f 0xa4 0xae 🤮 Stopped with slippage
0xf0 0x9f 0x99 0x81 🙁 Stopped
0xf0 0x9f 0x98 0x90 😐 Scratched
0xf0 0x9f 0x99 0x82 🙂 Profitable miss
0xf0 0x9f 0x98 0x80 😀 Hit 1st target
0xf0 0x9f 0xa4 0xa9 🤩 Hit 2nd target
0xf0 0x9f 0xa4 0x91 🤑 Hit 3rd target

Note the happy:sad ratio is 5:2, that is by design.

Regarding UTF-8 support, I may have found a bug in SierraChart - support ticket here describing the issue.

Started this thread Reply With Quote
  #18 (permalink)
 tklee 
Los Angeles California
 
Experience: Intermediate
Platform: NinjaTrader
Trading: Futures
Posts: 1 since Aug 2021
Thanks Given: 0
Thanks Received: 0

This looks amazing. I would love to beta test this, if you ever need.

Reply With Quote
  #19 (permalink)
 Futures Operator 
New York, NY
 
Experience: Intermediate
Platform: Sierra Chart, thinkorswim
Broker: Amp-Rithmic/TT, IB
Trading: CL, GC, NQ
Posts: 601 since Nov 2010
Thanks Given: 2,039
Thanks Received: 258

Would you be able to share this?

Reply With Quote
  #20 (permalink)
 shokunin 
Manchester, United Kingdom
 
Experience: Advanced
Platform: Sierra Chart
Broker: Optimus Futures, Rithmic, Denali
Trading: ES
Posts: 85 since Jul 2020
Thanks Given: 6
Thanks Received: 157


Wow, it's been a while!

I was offered some IT contracts when the Covid lockdowns finished. Due to time constraints, I was unable to daytrade or continue development.

I'm now back trading (part time), so will continue development. To get back into trading I've joined Earn2Trade, as it's been almost a year since I day traded. I might start a forum post about that if anybody is interested.

Started this thread Reply With Quote




Last Updated on April 5, 2022


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts