posted 10-06-2005 12:26 AM         

This has all been discussed before on this forum, and in considerable more detail than in your post.

Just for starters, see http://www.stock-anal.com/ubb/Forum1/HTML/000684.html

posted 10-06-2005 11:13 AM         

As you retest each day, all that B/C is doing is reoptimizing using the benefit of the latest series of back test days. What is wrong with that? B/T results are never precisely correct and this is reflected in this day to day change. Johnny, I would be interested in your experience using 250 days...

Regards,

posted 10-06-2005 11:35 AM         

(1) An indication whether each of the indicators is in a bullish or bearish mode.

(2) The buy & sell signals for each indicator for that B/T report.

Thanks for your consideration.

posted 10-06-2005 04:00 PM         

David
------------------------------------------

However, the novel idea of dynamic optimization needs to carry with it a few caveats. We need to understand the nature of the tool, and the processes, that we are dealing with.

The most important thing to realise is that the gain (or loss) percentages quoted by the optimizations represent the best possible return that could have been made, and can occur only if one had known in advance how to calibrate the indicators accordingly. For example, let us assume that the optimized RSI for the last 88 days is a 5 day RSI, with buy and sell signals being generated whenever this crosses a 7 day MA of itself. But the same 88 days worth of data was not available to BC on the days that the buy and sell signals, which generated this maximised return, actually occurred. On each of those days, BC would have been looking at a different (earlier) 88 day window, resulting in completely different optimization, indeed one which might not even have generated a signal. In other words, the optimizations are all being performed with the benefit of hindsight, and hence the returns quoted are a maximised, rather than a realistic, value.

But understand this – the above in no way undermines their usefulness. If we assume (and I would suggest that it is a reasonable assumption), that (1) each stock price action has its own distinctive ‘character’, or cyclical rhythm, (2) that this character is ever-changing (‘evolving’), and (3) that the most recent character has the greatest probability of imminent recurrence, then it follows that dynamic optimization is an smart and logical way of capturing, and then applying, this behaviour. So although the returns quoted should not be used as a realistic measure of past trading performance, the results of the profitability back testing do – in the context of the immediate future – represent a perfectly valid and objective means of (1) effectively calibrating these indicators, and (2) performing a head-to-head comparison of the potential value of each indicator, relative to the others, given the most current OHLCV data. But with one proviso (see below).

First, an aside. There are other reasons why the returns quoted should not be taken as indicative –

(1) The system makes no use of stop losses, or any premature full or partial exits used to progressively extract profits. In this regard, then, profits quoted by BC would likely be understated.

(2) The system also makes no allowance for transactional (or ‘dealing’) costs, which would potentially have the opposite effect, i.e. of overstating profit on every trade.

Given that BC is compounding gains and losses, both of these effects would be inflated exponentially.

The more trades that occur over the period being back-tested, the more frequent the dealing costs, and the greater the effect of inaccurately inflated profits. There are situations where BC optimizes an indicator to two very close calibrations (whose crossovers generate the buy/sell signals), e.g. an EMA(1) and an EMA(2) for the BBI, resulting in a large number (e.g. 50 or more) of trades. Given that the transaction cost occurs once per trade, then, when the head-to-head comparison of indicators is being performed, allowances need to be made for the number of trades. Hence, I believe that the ‘average percent gain per trade’ column should be read as being a more valid basis of comparison than the ‘total percent gain’, for each indicator.

I assume that the optimization process simply runs a set of values through each parameter associated with the indicator being optimized, and then calculates crossovers, buy/sell signals, gains (or losses) for each trade, and then compounds these for a total gain. Then this process is repeated for the next value, or combination of values, in the set. For example, to optimize RSI, BC would run the values 1,2,3, etc to a preset maximum through the RSI period, and then cross these with the values 1,2,3, etc through the MA being used for the crossovers. Thus the value pairs (1,1), (1,2) (1,3) … (2,1) (2,2) (2,3) … (3,1) (3,2) (3,3) … etc would be tested, and the compounded profitability calculated for each pair. The pair that generates the maximum profit becomes the optimized value. Of course I may be wrong, but that is my assumption as to how the optimization process would most likely operate.

Now for the aforementioned proviso, which also concerns the disregard of transaction costs. (This leads to the subject of an earlier post of mine on this forum). If the ‘maximum profit’ for all trades across the back-test period is used to determine the optimum RSI value, then this is not a true optimization. My view is that the average precent gain per trade would give a more realistic value. Note that I am not repeating myself here. A couple of paragraphs back, I stated that the AVERAGE percent gain per trade was a more valid basis of head-to-head comparison of the indicators. Now I am contending that this is also a more valid basis for calibrating the indicators during the optimization process itself.

Hopefully an example will clarify this further:
Supposing RSI(3,4) results in 50 trades generating a total (gross) return of 25% (average gain per trade = 0.5%)
While RSI(14,16) results in 4 trades generating a total (gross) return of 10% (average gain per trade = 2.5%)
Then my contention is that (unless one is trading commission free), RSI(14,16) is easily the superior calibration. However, the only way of accurately calculating this would be to have the user input a cost per transaction, which was the subject of my earlier post.

It follows that, when evaluating the chart-based output generated by BC, I tend to disregard the indicators that whipsaw in and out of trades every 1-2 days. For the reasons I have stated, I see this data as being aberrant and of little value.

Finally, some general caveats that don’t specifically relate to BC.

The fact that all indicators are ultimately derived from OHLCV values undermines, to some extent, any confirmation effect of coinciding indicator signals, especially where the indicators concerned are calibrated across a similar time frame. Offsetting this is the fact that BC uses a diverse smattering of price, volume, and momentum based indicators, all summarised into BBI, which is then itself optimized. So common sense should be used, to decide exactly how strong the confirming effect is.

Indicators are capable of forecasting the probable direction of a trend, but not necessarily its gradient, or future duration, which ultimately determine profit (or loss). Like other TA systems, BC will generate effective entry signals, but even the best entry signals offer no guarantee of profit. In general, an earlier entry will net one a better price, but with lower probability that a confirmed trend has been established. Profit is only achieved if the trend persists long enough to allow costs to be overcome, and exit occurs before the profit is consumed by the eventual reversal. Moreover, of the factors that ultimately determine one’s bottom line profit (e.g. entry point, exit point, stock selection, position size, etc), position sizing way outweighs the others in its significance.

posted 10-07-2005 04:46 PM           

I wanted to test this application on my 30 day trial so I just ran BT using Historical data. I went pt by pt running a moving window over a 60 point period. Shazam! There it was after every optimized run - BC uses the now current price at the time the security was loaded and today's date as the last entry for the "historic" B/S signal. For example, running on historical data truncated as if today was a close of Sept 1st, the now (delayed from Yahoo) price and today's date of Oct 7th is used as the last data point. That changes the last decision point hugely since the last data point is now and not Sept 1st. The equation is unbalanced.

It took a while to see this, so, I'd like to point this out to Mr. Admin. Do you have an explanation? I just can't find item in the posts.

posted 10-08-2005 07:38 AM         

A couple more points about AIO.

AIO can (and possibly should) be used as nothing more than a tool. There is nothing to prevent (1) experimentation to find a good set of fixed calibrations (e.g. RSI(14)), and then (2) running AIO periodically (e.g. when price behavior, cycles, etc change) to see if a better set of optimizations would now fit better. Just because AIO exists, it does not have to be used every day, in fact I would suggest that over-use reduces its meaningfulness. As with any charting method, it is important to understand exactly what is being measured, and how to meaningfully interpret and use it.

Remember also that indicators crossing their own EMAs is not the only way that they can be used. For example, oscillators (RSI, Stochastic, etc) can be used to highlight changes in momentum, and divergences, that are obvious simply by looking at the charts, i.e. completely independent of the buy/sell signals or the Bullish/Bearish count.

posted 10-08-2005 09:50 AM         

It was quickly evident that the 10 day period gave indicator parameters with a high INDICATED rate of return, but which rapidly changed day-to-day, and actually caused losses despite the high INDICATED gain in the B/T. This was also true, to a lesser extent, for the 88 day back tests. The 250 day test provided the best actual profits, despite the fact that the INDICATED profit per trade was less than for shorter back tests. It also changed the least with more recent sample series. In addition the 250 day period filters periodicities of 1 yr. or less to minimize their influence on the results.

As in many subjective techniques, one way of resolving this matter(the length of the B/T) is going to extremes in a trial & error technique-which often clears up the matter as it did for me in this instance.

EXPERIMENT with the 10 day period versus the 250(or 325) day period & see what you think. The equity should be one that has maintained its eyeballed(on a longer term chart) cyclic characteristics during up & down movements for the test period(like QQQQ does.)

My intuitive explanation for this is because of the increased number of trades in the longer period, giving statistically more consistent trades for the optimization. I am inclined to believe that the improved accuracy is proportional to the square root of the number of days in the sample. The premise here is that future trades will be indicated based on the essentially unchanging AVERAGE of the signal dynamics over the period of the B/T.

Since there is such a variance of opinion on this subjective matter, appropriate trial & error tests seems like a good way to better resolve the matter. It doesn't take long to see that a 10 day back test produces poor actual results, & changing B/S signals, when trades are made on its indications. LOL! Wotever ya do, don't just take my word on it....

Best wishes to all, & especially to David for his untiring & expert help,

posted 10-27-2005 03:48 AM         

I'm not sure whether you're saying that AIO is misleading in concept, or whether the software is not implementing it correctly (i.e. has bugs that need fixing).

Personally, I'm 50-50 as to whether AIO is more likely to generate more profitable trading signals, than simply using fixed parameters (i.e. Parameter Set 1). Because of its very nature, AIO will cause back-testing to generate different PAST buy/sell signals today than it did yesterday. That's because each day AIO is attempting to optimize by using a completely different data set (the oldest price bar is dropped from the set, and today's bar is added). The smaller the data set you're working with, the greater the propensity that these "signal changes" will occur (which is why I believe Frankenstein recommends a window as large as 250 days). If you feel that AIO is producing misleading results, then stay with fixed parameters (Parameter Set 1), which will ensure that the signals in the past remain fixed, and hence there can be no "cheating".

There are other reasons why back-test's past profitability is unrepresentative, and should therefore be used only as a basis of comparing indicators (as opposed to gauging potential future profit). Back-testing does not take transaction costs and slippage into account, which means that profits will be overstated, especially if the number of trades across the back-testing period is high. Conversely, BC does not allow setting stoplosses, and unrestricted losses on some transactions means that overall profit will be understated. Both of these contrasting effects are magnified in that BC assumes that capital will be applied exponentially (i.e. any gain or loss on previous trade is added to the capital applied to the following trade).

Let's suppose that back-testing shows that RSI(14) would have generated a profit of 18% if it had used across the last 250 days, where RSI(20) would have generated a profit of 45%. That is NOT trying to say that, if you set the parameter for RSI to 20 that you will likely make 45% across the next 250 days. It is simply saying that if the past cycles are repeated, that ON BALANCE OF PROBABILITY a parameter setting of 20 for RSI will work better than 14. Of course, there are no guarantees that this will be the case, just as there are no guarantees with ANY system. All technical analysis methods make some assumption that the most recent patterns, trends, support and resistance, are MORE LIKELY THAN NOT to repeat themselves in the future. Otherwise we might as well toss a coin, to decide whether a stock's price is more likely to rise or fall.

As I've stated elsewhere, BC uses some indicators (especially oscillators like SO, RSI, etc) in non-textbook fashion. Oscillators measure momentum and are generally highlight overboughtness or oversoldness, and also decelerations in a trend as divergences. In contrast, BC uses them to generate buy/sell signals when they cross over their own moving averages.

Returning to AIO, my preference would be to use it as a starting point to give initial indicator parameters. If the signals in the resultant indicator charts match (approximately) the cycles of the price chart, then I'd stay with the parameters for those indicators, otherwise I'd experiment with my own settings, which may or may be different from the factory defaults (Parameter Set 1). Once I've decided which indicators and parameters work best, then I'd tend to stay with them, until the stock is clearly cycling with a different rhythm, or the back-testing is producing decreasingly profitable results. I agree that performing AIOs too often can create confusing results.

The late Bruce Babcock, who was a successful commodity futures trader, was opposed to using optimized parameters. In fact, he believed using the same indicators and parameters across EVERY stock (or commodity).
See http://www.rb-trading.com/article3.html and http://www.rb-trading.com/article7.html

I don't think BC's "advertising" is necessarily misleading. Just because AIO is available doesn't mean that it must be used on an everyday basis. The documentation (and it is not that extensive) tells you more about which buttons to push, rather than how to interpret the meaningfulness of the results. Along with others, I have attempted to "fill these gaps" by posting thoughts and suggestions on this forum.

I am not trying to defend the software, as I have no stake in it, and nothing to gain by doing so. I'm trying to be helpful by correcting any misconceptions, and hopefully improving users' understanding of how to use it meaningfully, and interpret the results. At the end of the day, software is merely a tool, it's how you understand its output, and apply this in the overall context of your trading system, that is most important.

posted 10-27-2005 05:14 PM         

##A more correct statement is that AIO, as its name implies, optimizes the parameters of a Best-Charts indicator based on the selected last number of days of data. As David has explained, as you add data from a new day & drop off the data from the last day, the indicator is re-optimized for the most recent past.

I find it very useful. If I reoptimize and a confirming signal appears-no problem-it confirms the previous signal. If I reoptimize and it contradicts a previous signal that I have acted on, I consider it a signal and immediately liquidate the position as best I can.

It uses the immediate past as a model for the future. As I see it, there is no misrepresentation, and not a bug since the software is performing the task assigned to it. It should only be considered one of the useful tools used to buy & sell.

posted 10-28-2005 04:58 PM         

Well the buy signals have now disappeared from the past, since CNR did poorly yesterday. Should have just added a sell signal since a buy signal was already issued. The software is broken!! I’ll prove this weekend that the first data points don't matter (not a moving window effect!) and this is just a bug. BT is misleading; I purchased the software based on it’s results. I’ll also check BT without AOI and probably will find it changes it mind also. Since it changes its mind I cannot judge if BT is really a good tool. Seeing 12 buy/sell signals via BT when in reality it could be 40 is not okay.
##Interesting!...##

Most users are not day traders. I do not have the time to trade that often.
##I have said nothing about day trading and do very little of it.##

I’ll find out this weekend if BT is broken period. I believe this could be fixed and will have further comment on this after further testing.
##Fine-realizing that AIO is just optimizing each of the B/C indicators based on the data from the most recent sample period.##

posted 10-30-2005 05:50 AM         

I dislike salesmen, or more specifically their hollow sales pitch. This is the first time that I've ever been compared to one!

If BT "changes its mind" with fixed parameters (i.e. without an intervening AIO) then there is either a bug in the software, or the underlying data has changed on you. Unless you save data from elsewhere and then present it to Best-Charts as a historical file, Best-Charts defaults to Yahoo as its EOD data source. When I first used BC more than 12 months ago, Yahoo EOD data was fairly reliable, but I understand that currently this is not necessarily the case. The Yahoo data from certain countries' exchanges does not remain consistent.

I'm not defending BC. As I said before, I have nothing to gain by doing so.

David

posted 10-30-2005 02:21 PM           

I apologize on the salesman remark. It was bad judgement on my part. You two are truly professional and tolerate my new experiences with this tool.

posted 11-29-2005 10:28 AM           

When you run AIO, you're optimizing indicators for a "given set of data". That is, the indicators are changed to fit "how the market has been" over the data set.

We could describe the last nearly 2 years as "moderate volatility in a tight trading range"... running AIO right now would optimize indicators for THIS currenty type of market condition.

Should the market change to something different, like "low volatility, strong up", running AIO would reset the indicators for THAT period and those settings would be basically useless over the previous period of "tight range".

If you run AIO over a shorter period and where the market's character has changed somewhat, back testing over the period BEFORE the market's character shifted renders all B/S signals worthless.

With this in mind, you should better undertand the limitations of indicator optimizing.