The Optimization Honeypot

by Bill Brower on September 1, 2016 | Categories: Uncategorized |

The Optimization Honeypot

Black hat hackers seeking computers for the purpose of enslavement in their bot army look for unprotected or poorly protected machines that allow easy entry of malware and Trojan programs. The white hat hackers actually invite these malware programs into specially configured computers to catch the malware so that it may be examined and defeated. They call those machines honeypots.

The Tradestation strategy optimizer can behave like a honeypot trapping the unaware user by providing the appearance of astonishingly good results. I have seen this happen many times over the years. I even have one client who has spent years modifying and tweaking the very same strategy and running it through the optimizer attempting to eke out slightly better performance. Yet every time he tries to trade the strategy, he quits in frustration at the subpar performance. The strategy takes literally thousands of trades, so there is no problem with curve fitting issues. The problem is that the average profit per trade on the S&P 500 E-mini is only $15, and that is without any allowance for slippage. He argues that there is no slippage because he uses limit orders for entries and exits.

The problem is that there is always slippage even with limit orders. Slippage is a catchall category to allow for unexpected events, not just to provide for the difference between where you thought you were going to get filled and the actual fill. Slippage must cover failing to get filled at your limit order price to exit a profitable trade. It covers the failure to get filled on entries that clearly moved to your profit target exit. It also covers data errors, internet connectivity issues, and Tradestation execution issues. So in reality my clients program which shows phenomenal profitability in the strategy performance report, is really phenomenally unprofitable.

He has been caught in the optimization honeypot. When that happens, the results appear irresistibly good. The reason this happens is because when the optimization is run without allowance for reasonable slippage, Tradestation identifies parameter settings that maximize net profit, at the expense of minimizing average profit per trade. So for years my client has been focusing on tweaking parameter settings that generate a mirage of high net profit. If he had instead included some reasonable slippage, the optimizer would have directed him toward completely different parameter settings. This would have had a dramatic effect on the actual strategy development because he would be focusing his efforts to maximize the performance of a completely different set of trades than the ones he has been looking at. I am not only talking about a vast amount of time and effort on his part but also significant investment cost in programming his strategy. His explanation was that he made an overall allowance for slippage at the end of his optimization and this was the same thing as allowing some slippage on every trade. That was his fatal flaw. They are not the same. There is no way for him to be directed to an alternate set of parameter settings so his focus for strategy development was always in the wrong direction.

The optimizer is indeed a very powerful tool and very useful in the right hands. Like any tool it is open to abuse and misuse. I think perhaps more often than not it is misused by most who have ever worked with Tradestation developing strategies. Most of them are no longer trading or even using Tradestation. Don’t allow yourself to be mesmerized by bogus results generated by the optimizer. Always allow for reasonable slippage even if you are using limit orders.