December 5, 2012

To make the concepts in my trading methodology clearer and mainly to answer some recent criticism concerning the mathematical expressions used in my notes, short of maybe giving my programs away, I opted to revisit the foundations on which my methods rest and explain them in more detail. 

My trading methodology is best explained using mathematical equations and I find them necessary to represent my trading models. These models are not new, they have been around for quite a while; some over 100 years. I'll try to set the basics straight and later proceed to what should have been part 2 of winning the game.  

I have expressed this before many times. What I advocate is a trading methodology based on the simple idea that one can trade over a stock accumulation process and by doing so generate better results than using a Buy & Hold strategy or a trading only strategy. That is the basis of my trading methods: instead of doing one or the other separately, you do both.

It is not very complicated, there are not that many ways to express this. While you accumulate shares for the long term, you can trade short-mid-long-term over the process and use the proceeds to accumulate more shares which might or might not stay in your portfolio depending on if they are sold or not down the line. This creates a feedback loop where trading profits can be re-invested to accumulate more shares with for side effect an increase in long-term performance.

Once that is said, there are not that many options: either it is an opinion which does not need proof and as such does not matter much, or it can be expressed as mathematical statements that at least should be demonstrated to work by simulation over real stock market data. And whatever mathematical expression used, it better hold with an equal sign, not a fuzzy: maybe I think that or I feel like or in my opinion this or that kind of thing.

I would have thought that after 4 years of presenting the same basic idea most people would have caught on, after all, how complicated can it be? But it might appear that I underestimated how anchored some beliefs and notions are. There are really two camps out there, each looking at the same problem but with a different perspective and with practically no one holding middle ground.

My research shows that both fundamentalists and technical traders can benefit by using a stock accumulation program to which is added a trading component. Each school of thought has separately about the same long-term expectancy and if combined should provide an additive return mix: %R = %F + %T.

My observation has been that both sides can increase their performance levels by borrowing some traits from each other's methodology, and by doing so combine to provide a multiplicative return mix: %R = %F x %T. In the beginning, it does not make much difference, but with time, adding or multiplying two exponential functions can mean all the difference in the world. The impact is on the doubling time and it can prove to be considerable.

The Basics

No matter what stock trading strategy one wants to use, it can be explained by the cumulative history of all the trades taken. In the end, all the profit or loss generated over the entire life of the portfolio will be represented by a single number: the ending portfolio value.

The trading history, as an evolutionary process, spans the whole life of the portfolio and should be viewed in its entirety: from start to finish. It is the sum of all the trades, good and bad, which will determine the final outcome. And to reach the finish line; one has to take every step along the way and put in the time to get there.

There is not that much one can do playing the game: stocks only go up or down and you can only take a position or not, and determine its size. There are variations but they all translate to being in a position, long or short, for some duration.

A stock wealth accumulation process can be described using only a few variables such as price, quantity and time.

A price series can be expressed as a function P(t) having an initial price Po  to which is added all its future price variations, up or down, as in the following:

P(t) = Po + ΣΔP                                                                   (1)

where ΣΔrepresent the sum of all price variations and where ΔP =  Pt - Pt-1  is the difference in price from one period to the next.

The difference between the end and starting price: Pt - Po is the sum of all price variations in between up to time t: ΣΔP, and if there is an appreciation in price over the whole trading interval, it will be because of ΣΔP  > 0. Meaning that the end price is greater than the initial price: Pt  > P. Equation (1) does not say how or why the price reached its final point, only that whatever the path was, and whatever was the observation period the end price is the sum of all its past price variations.

There are many other ways to express the same thing. This model is simple and is able to represent any stock price over any time interval. It does not say what was the path taken or how it zigzagged from start to finish; only that whatever the price variations over the interval, the price function gives the correct answer.

Another convenient price function is the compounding rate of return over time representation: P(t) = Po(1+r)t where the initial price Po appreciates at a rate of return r over the trading interval t. Both representations give the same final answer.

The same mathematical structure as (1) could be applied to a portfolio wealth accumulation process resulting in:

W(t) = Wo + ΣΔW  

where the wealth equation would start with an initial capital Wo to which would be added the sum of future portfolio variations ΣΔW. The objective, in a portfolio management setting, is to achieve the highest wealth possible under the constraints of a limited initial capital and almost unpredictable stock prices. Another view of the wealth function is the initial wealth to which is applied a compounding rate of return as in W(t) = Wo(1+r)t.

One could also consider an asset valuation function as an ongoing summation process dependent on the integral of a continuous quantity function over the price gyrations for a trading interval as in:

 A(t) = Ao   + ∫t Q(t)dP 

The above equation would resolve the investment in a single asset as a linear function and would over time represent the payoff of the trading strategy Q(t) which would be added to the initial purchased asset Ao. This representation of a time series is again one among others. The model only serves to describe the price movement between two endpoints. For a trader or investor, Q(t) can represent a series of entry and exit points and the history of the inventory held over the investment period.

For a Buy & Hold scenario where the initial quantity of shares purchased would stay constant over the duration of the investment period, the above equation would be:

A(t) = Ao   Q(t) ∫t dP 

which clearly shows the inventory as a constant over the trading interval, and therefore performance (the payoff) would depend entirely on how price behaved over the integration period.

The Payoff Matrix

In the same way, one could attempt to encompass all the trading in a portfolio of stocks over a trading interval using matrices. One of my preferred expressions is:

W(t) = QoPo + Σ(H.*ΔP)

which is equivalent to the integratable form but using matrices of discrete values. And where QoPo represents the initial stock purchases: the initial number of shares Qo acquired in each stock i at prices Po. And where Σ(H.*ΔP) stand for the payoff matrix: the total profit generated by whatever trading strategy H applied to the stocks in the portfolio. H is the matrix of the cumulative stock inventory on hand for each security over the portfolio's lifetime.

It would have the Buy & Hold scenario:

W(t) = QoPo + HoΣ(ΔP

The price variation matrix ΔP is simply the price difference from close to close (or from period to period) of each security. H and ΔP are n x m matrices where each column m represent the time series of a particular stock and where each row n stand for the daily price change or any other trading interval of choice. Therefore, each column m is the representation of a single stock's price series as it develops over time, as expressed in the beginning as P(t) = Po + ΣΔP

In my Changing the Game note, I provided examples of the H and ΔP matrices using Excel.  Here it is again:

   Excel Matrices

For a full-size image, use this link: Excel H and ΔP Matrices and zoom in to see the details.

As can be noted in the above graph, summing the columns of the (H.*ΔP) matrix gives the cumulated profit or loss generated by each of the m stocks in the ΔP matrix.

What does it all say?

Any portfolio of stocks is just a bunch of stocks - a ΔP matrix - which can represent the price variations for any set of stocks over any duration. Take for example the 100 stocks of the S&P 100 over 20 years. The ΔP matrix would show all 500,000 closing price variations over the chosen trading interval (5,000 trading days). The H matrix would be the day to day account of the number of shares held in each of the stocks including, at times, zero for no inventory or negative quantities to indicate shorts. The inventory matrix H is the running total of shares held in each stock over the whole trading interval. It has the same size as the ΔP matrix.

To represent a single trade, to get the profit or loss generated, one can use:  

Profit  = Q (ΔP)      ≈   Q (Pt - Po)

where the profit or loss is the difference in price ΔP between the buy and sell price multiplied by the number of shares held Q. The payoff matrix can do the same in a single operation and evaluate all the generated trades in a portfolio.

The H matrix entries have for each element the number of shares held (see the above graph). Each column of the H matrix maintains the inventory level of each stock over the trading interval. Again, for the S&P 100 ΔP matrix example, the H matrix would have 500,000 entries representing the ongoing inventory level in each of the 100 stocks over the 5,000 trading days. This also means that 500,000 decisions, to trade or not to trade,  had to be made to determine inventory changes over the life of the portfolio.

The payoff matrix represents the most concise mathematical expression to convey the outcome of a trading strategy. It is the discrete form of the integral presented earlier.


The (H.*ΔP) matrix is the mathematical representation of an element-wise multiplication ( .* ) meaning that the multiplication is done element by corresponding element, where each element is the profit or loss for that stock on that day; it could be expressed as:

Hi,i · ΔPi,i  

The above product gives the profit or loss on a single position Hi,i in a single stock for a particular day ΔPi,i. And for the payoff matrix, the 5,000 trading days by 100 stocks would show the 500,000 entries showing the profit or loss for every single day for each of the 100 stocks. 

The summation Σ(H.*ΔP) is done over all columns in order to provide the accumulated profits generated by the ongoing inventory level of each stock. And summing the last row will provide the entire portfolio's total generated profit or loss.

A Single Expression

In a single expression, one can describe any trading strategy, its whole history and its outcome: the total generated profit or loss. It enables to view the entire life of a portfolio and its inventory movements. It also forces one to view a portfolio over its entire history; and not just something that goes from one trade to the next.

I haven't seen a simpler representation for a total portfolio. One expression: Σ(H.*ΔP) saying it all. You are not only viewing a trading strategy but the evolution of a complete portfolio over its entire trading history in one shot.

What else is being said by the payoff matrix? First, whatever the stock selection, the ΔP matrix is set. Anyone selecting the same set of stocks over the same time interval has the same ΔP matrix to contend with. And if one has a better trading strategy, it will be because of how the inventory levels were handled:

Σ(H1.*ΔP)  > Σ(H2.*ΔP)     H1 > H2

It becomes easy to compare trading strategies over the same ΔP matrix. For a strategy developer, it will also be his/her quest; finding better and better trading strategies. With the payoff matrix, one can view the whole portfolio management process as a monolithic block representing everything that is being done over the entire life of the portfolio over every single stock.

Strategy design changes perspective, it goes from linear day to day functions to vector and matrix manipulation functions. From single stocks to multiple stocks scenarios over the entire time horizon of a portfolio's life; that it be over past data, or on future data that will continue to expand in time.

The payoff matrix can represent any trading strategy whatsoever. In its simplicity, it does not bother with predictions, momentum or time even though time is an integral part of the expression. It is only concerned with what happened or will happen.

The payoff matrix deals only in trading results. For the life of the portfolio, has there been a profit or not? Which can be easily expressed as:  

Σ(H.*ΔP) > 0

If your trading strategy cannot achieve this simple objective, meaning being profitable, then why would you want to undertake executing that strategy at all? Look also at all the time that would be wasted.

If for whatever trading methods you have used over the portfolio's life you could not show a profit for your efforts; then it can only demonstrate that your trading strategy is not worth that much. One could even venture that it is detrimental to your portfolio's health. And should your trading strategy do less than what the Buy & Hold strategy could do, then again you wasted your time pursuing a strategy that could have been achieved with practically no effort at all.

What else can be extracted from the payoff matrix?

For one, its size matters. Take a 10 x 10 payoff matrix: 10 stocks by 10 trading days. Anyone testing on such a limited set over such a limited time interval would immediately conclude that whatever optimized test you may have performed is totally irrelevant. There is nothing you could extract from such a backtest, and certainly, nothing that could be useful in determining what would be the outcome executing that strategy over say the next 100 or 1,000 trading days.

If such is the case, then how about 10 stocks by 1,000 trading days (about 4 years) which would be about the historical duration of an average economic cycle. Increasing the trading interval 5-fold would include more market cycles and result in 5,000 trading days amounting to a time horizon of some 20 years. The longer duration would provide sufficient time to show a trading strategy's merits, meaning generate profits under varying market conditions and would also represent a more than sufficiently long trading interval to show statistical significance.

However, even a payoff matrix of 5,000 trading days by 10 stocks should be considered insufficient as testing ground for lack of generality. Having a trading strategy which would have a positive payoff matrix over some 10 stocks is no guarantee that picking another group of 10 stocks would also show a profit. To achieve statistical significance, one would have to increase the sample to over 30 stocks. The reason is simple, it is above this sample size where statistics gathered can have some significance. A portfolio starts to be considered diversified if it has more than 30 stocks. And in the market, the more you diversify, the more the average price of your stock selection will tend to highly correlate market averages.

A backtest over some 20 years may take a few minutes to execute on your computer. However, a 20 years forward test will require over 5,000 trading days or some 7,304 calendar days. And forward testing could represent a lot of wasted time; especially in a compounded return game. It becomes preferable to make a lot, and I mean a lot, of backtests to show the merits of your trading strategy than applying it blind on future data.

You backtest over some 20 years to show the merit of your strategy. Then you are required to forward test for the next 10 or more years to again show that there was some merit in your approach. I even had someone say: forward test for the next 3,000 years, then I might consider your approach. To which I replied: please don't hold your breath.

To make the data sampling more significant, one could consider using the 100 stocks of the S&P 100 over 5,000 trading days. This would result in a payoff matrix with 500,000 data elements in each of the H and ΔP matrices with for unique objective, over those 20 years, to generate a profit:  

Σ(H.*ΔP) > 0

This would not change the nature of the payoff matrix, the sum of the element-wise multiplication would remain the correct answer, the size of the matrix is not a constraint. It still summarizes the total portfolio history of the sample being used.

But making a profit is not enough, the trading strategy must also provide better results than the Buy & Hold:  

Σ(H.*ΔP)   >  Ho Σ ΔP

And whatever this trading strategy H may be, it will have to view the problem not from the perspective of a single stock, but for a portfolio of 100 stocks over the entire 20 years' duration of the investment period. Backtesting on too small a sample or for too short a trading interval can only decrease the confidence one might have in his/her trading methods. The same would be said about over-fitting or over-optimizing the trading strategy. Some compromises could be considered, like say 100 stocks by 1,000 to 2,000 trading days to get a feel for a specific strategy. One could also increase the number of stocks to make the sample selection even more representative.

Recapitulating, the payoff matrix is sufficient as an expression to analyze the outcome of any trading strategy over any stock sample. And by doing backtests on significant data samples, one can design better trading strategies that could be applied to future data. It is a given that if your trading strategy cannot even backtest, in the sense that it can not even generate a profit over past data of significant size and duration, then there is no logical reason to assume that the trading strategy would do any better in the future.

Having a trading strategy that can be applied to only a few stocks over a few years does not seem to be a reasonable view of the portfolio management problem. And whoever wants to demonstrate that his trading strategy, spanning over a single stock over a single year, is superior should seriously consider expanding the size and range of the data set to be backtested, to say the least.

There is, as in anything else, some common sense to be applied.

Looking forward, say for the next 20 years, I have no idea how the 100 stocks of the S&P 100 will evolve over time. Which are the ones that will most benefit or which will go bankrupt? The whole ΔP matrix is an unknown. It is up to me to design a trading strategy H that will evolve in time to extract as much profit as I can with whatever trading methods I have. But one thing is sure, whatever the strategy I may use, it will have to at least outperform the Buy & Hold, or else...

... (to be continued)...

A preview for Winning the Game II

The inventory matrix H can be controlled to do what you want. You decide when and what quantities to buy, sell or hold depending on your decision surrogates. This decision process will govern how the inventory will evolve over time:  

H = Ho .+ B . - S

where the inventory H will be the result of the initial positions taken Ho to which will be added element-wise the quantity of shares bought B minus the quantity sold S. The B and S matrices are of the same size as the H matrix which for the S&P 100 example would require 500,000 trading decisions each.

The point will be to look at functions that can affect these trading decisions for the whole matrix, not just a trade or a stock, but the entire trading history. How to know in advance that your trading strategy will do better by mixing long and short-term trading ideas than standing alone.

Created... December 5, 2012,    © Guy R. Fleury. All rights reserved