Long-Term Backtests

Discuss your favourite picks, broker, and trading or investment style.
Post Reply
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Long-Term Backtests

Post by DenisD »

Two years ago, I upgraded my subscription at P123 for a month so I could do backtests from 1999. I created a bunch of pivot tables from the backtests. I was going to post some on FWF. But I never got around to it. Now's the time.

The first spreadsheet is for my US small value screen. It's similar to O'Shaughnessy's Trending Value strategy but with small caps. First, it ranks stocks with a six-factor value composite. Then it buys stocks from the best decile with the highest six-month return.

I ran backtests starting in January, March and May of 1999 with 10, 15, 20 and 25 positions refreshing every three and six months. So 3 * 4 * 2 = 24 backtests. I combined the results from all 24 into one spreadsheet with three pivot tables and associated pivot charts. The tables show averages of 12, 60 and 120 month rolling returns. They're set up to show the return differences between the different start times (January, March, May) and refresh periods (three or six months).

The worksheet, GainA, shows the yearly average of screen returns and benchmark returns. ExcessA, shows the yearly average of screen returns minus benchmark returns. GainExcessA shows the yearly average of excess screen returns for different benchmark return ranges and the count of the return values within each range.

There are some abbreviations: BM – benchmark, JA – January, MR – March, MA – May. I've set the spreadsheet to show 60 month rolling returns for 20 position screens.

Here's a picture of the ExcessA pivot chart:

USSV BT PT 2000 - 14.jpg

Each year has 6 vertical bars, one for each of the 3 start time/2 refresh period combinations. Each bar is the average of 12 five-year excess returns, one for each month.

What can we tell from this picture? Well, it definitely looks like refreshing every 3 months performs better than refreshing every 6 months. The different start dates lead to significantly different returns. Most of the bars are positive. But there's one discouraging trend: the height of the bars is definitely on a downward slope. Excess returns in the first few years are in the 30 – 40%/year range. In the last few years, they're around 5%/year. Some would say this is an example of the efficient market at work. :wink:

Clicking on the spreadsheet read-only link should put you in Excel Online. You can change the parameters or the structure of the pivot tables. But you can't save your changes, I hope. If you want your own copy you can download the spreadsheet or, if you're logged into your Microsoft account, copy it to your OneDrive. I'm not sure if the spreadsheet will work in older versions of Excel or compatibles.
Park
Veteran Contributor
Veteran Contributor
Posts: 1724
Joined: 13 Aug 2010 20:51

Re: Long-Term Backtests

Post by Park »

Thanks for posting this. I'm busy at work, so I don't have a lot of time to go over this.

http://blog.alphaarchitect.com/2017/04/ ... gs.wfHoLJo

In the above link, it gives results for US value stocks, as defined by Fama French criteria, from 2002-2015. The CAGR is 5.64%. For S&P500, it's 6.49%. So the results from 2007 onwards are in part explained by this.

Perhaps markets are becoming more efficient, which means decreased value and momentum premia. But I doubt that human nature is going to change that much. IMO, a large part of the value premium is d/t bubbles. Bubbles have always existed, and if I had to hazard a guess, always will.

If the value and momentum premium go to zero, gross returns on value and momentum should equal an index portfolio. So as long as one keeps costs under control, you shouldn't do too badly with a value and momentum strategy. And IMO, you've purchased an insurance policy against bubbles, which can hurt returns for many years.

Asness et al published an article on the value premium. IIRC, they found that even if the value premium was zero, there was still a benefit from it. That's because it diversified a portfolio. However, that assumed that one engaged in rebalancing between an index portfolio and a value portfolio. IOW, it was due to the rebalancing bonus.
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

Park wrote: 26 Apr 2017 08:11If the value and momentum premium go to zero, gross returns on value and momentum should equal an index portfolio. So as long as one keeps costs under control, you shouldn't do too badly with a value and momentum strategy.
That's what I thought. It would be an interesting project. There'd be a fair chance of outperforming significantly. And, even if I underperformed, it wouldn't be by much. So what happened?

I've ran a 20 stock small-cap value portfolio for 11 calendar years: 7 using Reasonable Runaways, 3 using Trending Value and one for the changeover. The following table compares returns of a Russell 2000 ETF, Vanguard mid-cap and small-cap ETFs and my screen:

Code: Select all

                  IWM       VO       VB   Screen
RR 7 Years        2.5      2.7      3.7     -2.1
TV 3 Years       15.4     16.4     15.7     21.9
All 11 Years      9.5      9.6     10.3      8.4

        2006     18.4     13.9     16.0     17.6
        2007    -16.2     -9.7    -13.9     -0.3
        2008    -18.5    -28.5    -21.4    -26.5
        2009      9.7     21.2     17.6    -13.8
        2010     20.0     18.8     21.0      9.8
        2011     -2.0      0.3     -0.5      3.8
        2012     13.9     13.5     15.7      1.8
        2013     48.4     44.5     47.3     55.9
        2014     14.5     24.1     17.3     41.6
        2015     14.1     17.7     15.0      2.9
        2016     17.7      7.9     14.8     24.3
Looks like RR underperformed by about 5%/year over the 7 years. Certainly more than I expected. So far, TV has outperformed by about 6%/year. Excellent! But there's that one bad year. And it's doing poorly again this year. Oh well, I'll be getting rid of all of those dogs when I refresh in a couple of weeks.

Overall, I'm down by 1 – 2%/year. But I have high hopes for the future! And my large-cap screens have more than made up the deficit.
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

First, I should clarify one little detail in my first post. When I say a 3 or 6 months holding period, I really mean 13 or 26 weeks. Similarly, 1 year is 52 weeks. Maybe I'll change that in the next edition of my spreadsheets.

The second spreadsheet contains more backtests of the same US small value screen. But this time they're rolling backtests. That is, the screen is run every week from 1999/1/4 to near the end of 2014.

Here are excess returns averaged by year and by month:

USSV RBT PT 66 1999-2014 by Year.jpg
USSV RBT PT 66 1999-2014 by Month.jpg

What does the spreadsheet tell us about the strategy for this period? Again, except for 1999, the best years relative to the benchmark were the early years. Excess returns got better as market cap decreased and number of positions went down. But volatility went up as excess returns improved. There was good outperformance in corrections. But slight underperformance in the worst bear markets. Grouping the excess returns by months rather than year shows that it was best to buy the screen in the few months before or after year end. The worst time to buy the screen was in the summer.

As always, future returns may not be as good as past returns. Even filtering out the first four or five years might show a different picture.

There are nine rolling backtests: three market cap rank ranges, each with three number of positions. The market cap rank ranges are 10 – 50, 15 – 67 and over 20. The smallest range has a lower liquidity requirement. The number of positions are 15, 20 and 25. The holding period is 26 weeks.

There are four pivot tables and their associated pivot charts on the Tables worksheet. The first table compares the average 26 week screen return for each year with the benchmark return. The second table shows the standard deviation of the screen and benchmark returns for each year. The third table shows the average of the excess screen returns for each year. The fourth table shows the average of excess screen returns for different benchmark return ranges and the count of the return values within each range.

As usual, there are some cryptic abbreviations. For the seven character column labels, the first three characters (S50, S67, All) give the market cap rank ranges from smallest companies to largest. The next two characters give the number of positions. The sixth character can be R – return or X – excess. The last character can be A – average or D – standard deviation. The three character column labels beginning with B are benchmark numbers with the last two characters the same as above.
Last edited by DenisD on 30 Apr 2017 22:52, edited 1 time in total.
User avatar
ghariton
Veteran Contributor
Veteran Contributor
Posts: 15954
Joined: 18 Feb 2005 18:59
Location: Ottawa

Re: Long-Term Backtests

Post by ghariton »

Thank you for sharing this.

George
The juice is worth the squeeze
Park
Veteran Contributor
Veteran Contributor
Posts: 1724
Joined: 13 Aug 2010 20:51

Re: Long-Term Backtests

Post by Park »

http://investorfieldguide.com/2014115ho ... portfolio/

"I set up portfolios which bought the absolute cheapest stocks trading in the U.S. (including ADRs). Portfolios ranged from 1 stock to 100 stocks, and stocks needed to have a minimum market cap of $200MM (inflation adjusted). Cheapness is defined as an equal weighted combination of a stock’s price/earnings, price/sales, EBITDA/EV, Free Cash Flow/EV and total (shareholder) yield. Each portfolio was rebalanced on a rolling annual basis (meaning 1/12 of the portfolio is rebalanced every month. Think of it like maintaining 12 separate, annually rebalanced portfolios). This means that the “one stock portfolio” will have more than one stock, because different stocks rise to the top through the months. This process removes any seasonal biases and makes the test more robust.
Here are the results, including return and Sharpe ratio. The best returns came from a 5 stock (!) portfolio. The best Sharpe ratio came from the 15 stock version. Both return and Sharpe degrade after 15 stocks." His data is from 1964-2014.

Other than Denis' data, this is the best data that I've seen on portfolio size. I certainly wouldn't recommend anything less than a 15 stock portfolio. When portfolios get this small, you're taking on a lot of volatility. But it may be relevant to a Canadian investor. Because of the size of the Canadian stock market compared to the US, a more concentrated portfolio might be considered. And the data doesn't take into account momentum, whereas Denis' data does.
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

I do a similar thing with my large-cap screens. Each one has two 10 stock screens refreshed six months apart and held for one year. I started out the same way with my US small-cap screen. But I changed it to one 20 stock screen held for six months.
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

A few more rolling backtests of my US small-cap value screen. They both have a market cap rank range of 10 – 50. That is, the market cap is less than the median. And the holding period is 26 weeks.

The first spreadsheet varies the number of positions: 1, 5, 10, 15, 20, 25. The average excess return of the 1 stock screens was 13%/26 weeks. For the 25 stock screens, it was 7.4%. The averages for the last 10 years were 5.9% and 3.6% respectively.

Here are the worst and best 26 week returns (not excess returns) of the 1 and 25 stock screens:

Code: Select all

Date      1 Stock Date     25 Stock
08/09/08    -94.1 08/09/08    -60.6
08/06/02    -84.2 08/09/02    -54.2
14/07/21    -82.7 08/09/22    -49.1
14/07/28    -82.0 08/08/25    -47.4
01/06/04    -78.0 08/09/15    -44.9
01/10/15    252.5 01/10/15     67.3
01/05/07    278.6 09/03/30     72.7
09/05/11    302.2 09/03/02     74.5
09/05/04    305.3 09/02/23     79.3
01/04/09    342.8 09/03/09     99.9
I wonder what the maximum drawdown of the 1 stock screens was.

The second spreadsheet looks at some variants of the strategy. They all hold 20 stocks.

S506 buys stocks with the best 6-factor value composite. It doesn't consider momentum. The average excess return is 4.9%/26 weeks, the poorest of the variants. It underperformed the benchmark by more than 10%/26 weeks in two years. But was the best by far in 2009 and the best over the last 10 years of the test. It does well in December but poorly in the summer. Benefits from tax loss selling, I presume.

S506F buys stocks with the best 6-factor value composite with 3 and 6 month momentum greater than the median. The average excess return is 7.8%, the highest by a small amount.

S506M buys stocks within the best decile of the 6-factor value composite with the highest 3-factor momentum composite. The momentum factors are 3, 6 and 9 month return. It's the third best with average excess return of 7.1%.

S5066 is my standard strategy. It buys stocks within the best decile of the 6-factor value composite with the highest 6 month return. It's the second best at 7.6%.

S5056 is the same as S5066 except it uses a 5-factor value composite. This Is the one O'Shaughnessy seems to favour these days. But the excess return was only 6.1%. Maybe I implemented it wrong.

That's it for the US small-cap backtests. I still have US large-cap and Canadian backtests. If anyone has ideas for better ways to display or interpret the data in these spreadsheets, please speak up.
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

Recently, I upgraded my membership at P123 for a month so I could do some long-term backtests. I wanted to evaluate some changes to my small-cap screens. I thought I'd post some of the runs here.

First, some rolling backtests of Canadian large caps for Dogs of the TSX fans. They all ran from the beginning of 2006 to near the end of 2016. Every week, 10 or 20 stocks were picked from the top 60 by market cap and held for one year.

I tested two strategies: pick the stocks with the highest dividend yield and pick the stocks with the highest shareholder yield (buyback plus dividend). There were some variations of each strategy, described below. I summarized the results over three periods: 2006-16, 2006-11, 2012-16.

Here are the results:

Code: Select all

            Dividend Yield             Shareholder Yield          Difference
Label   2006-16  2006-11  2012-16  2006-16  2006-11  2012-16  2006-16  2006-11  2012-16
NN10        2.5      4.1      0.5      3.4      3.2      3.7      1.0     -0.8      3.1
NN20        1.9      2.0      1.8      2.3      1.4      3.3      0.3     -0.6      1.5
NY10        2.2      4.1     -0.2      3.3      3.4      3.2      1.1     -0.7      3.4
YN10        2.3      2.4      2.0      3.6      2.4      5.0      1.3     -0.1      3.0
YY10        2.1      2.7      1.4      3.6      2.8      4.5      1.4      0.1      3.1
S210        1.4      3.2     -0.9      2.6      3.1      2.0      1.3     -0.1      2.9
S310        2.3      4.0      0.3      3.3      3.3      3.4      1.0     -0.7      3.1
The first two characters in the label column are:

NY – skip the worst decile of EBITDA / Enterprise Value.
YN – skip the worst decile of 13 week and 26 week momentum.
YY – skip both of the above conditions.
NN – don't skip any of the top 60 stocks.
S2, S3 – don't skip any but limit stocks/sector to 2 or 3.

The last two characters in the label column are the number of stocks, 10 or 20.

The first six columns of numbers are the average of excess returns of dividend yield and shareholder yield over XIC for the three periods. The last three columns of numbers are shareholder yield minus dividend yield.

I was a little surprised by some of the results. Shareholder yield did about 3% better than dividend yield over the past five years and a bit worse over the first six years. I was expecting to see the reverse. I wasn't surprised that 10 stocks beat 20 stocks in every period but one. Removing the worst value and/or momentum deciles was inconsistent. Sometimes a bit better, sometimes a bit worse. Limiting stocks/sector was pretty well always a bit worse. And 2 stocks/sector was worse than 3.

Here is a summary of excess returns for different XIC return ranges:

Code: Select all

            Dividend Yield                      Shareholder Yield
Label      <-35 -35 to 0  0 to 35      >35     <-35 -35 to 0  0 to 35      >35
NN10        7.6      2.3      1.7     17.6      8.4      3.1      3.5      0.8
NN20        8.4      3.3      1.2      2.5      4.2      2.8      2.2     -2.6
NY10        8.6      2.8      1.3     12.3      8.4      3.3      3.4     -1.0
YN10        6.6      4.3      0.9     10.9      7.3      5.5      3.0     -3.5
YY10        8.1      4.9      0.7      6.5      7.4      5.6      3.0     -3.3
S210       14.9      4.1      0.6    -13.5     12.7      3.3      2.7     -9.9
S310       14.3      4.2      1.8     -7.4     10.7      3.5      3.6     -7.8
Count         9      155      386       18
All of the extreme returns started before or after the 2008 financial crisis. Here we see the benefit of limiting stocks/sector. Those strategies did the best during the extreme low return years. But they did the worst during the extreme high return years. Of course, I only backtested through one financial crisis. Who knows if the results will be similar during the next.

If there are any gluttons for punishment out there who want to look at the details, here are the spreadsheets containing pivot tables and charts averaged by year and by month.

Clicking on one of the links should put you in Excel Online. You can change the parameters or the structure of the pivot tables. But you can't save your changes, I hope. If you want your own copy you can download the spreadsheet or, if you're logged into your Microsoft account, copy it to your OneDrive. I'm not sure if the spreadsheet will work in older versions of Excel or compatibles.

There are too many backtests in the spreadsheets for them to be of much use. It's probably better to just delete the strategies you're not interested in.

There are four pivot tables and their associated pivot charts on the Tables worksheet. The first table compares the average one year screen return for each year with the benchmark return. The second table shows the standard deviation of the screen and benchmark returns for each year. The third table shows the average of the excess screen returns for each year. The fourth table shows the average of excess screen returns for different XIC return ranges and the count of the returns within each range.

For the 10 character column labels, the fourth character is D – dividend yield or S – shareholder yield. Characters 5 and 6 are described above. The next two characters give the number of positions. The ninth character can be R – return or X – excess. The last character can be A – average or D – standard deviation. The three character column labels beginning with B are benchmark numbers with the last two characters the same as above. Column labels containing All60 have values for all of the top 60 stocks.
nisser
Veteran Contributor
Veteran Contributor
Posts: 2079
Joined: 11 Nov 2007 21:24

Re: Long-Term Backtests

Post by nisser »

When did you decide to go "live" with a simulation? I've come up with a sensical screening formula that outperforms the TSX by 20% annually. I've done every permutation when it comes to holding period (4 to 52 weeks) and even time brackets going from 1999 to 2017.This is a sample of the test periods and results. The first value is the excess annualized return, the second value is the maximum negative drawdown. So from 2005-2010 it maximally dropped by less than 15% than the TSX

1999-2017 Plus 17, -10% draw
1999-2003 Plus 13, -27% draw
1999-2005 Plus 18, -26% draw
2005-2010 Plus 16, -15% draw
2010-2014 Plus 24, -2% draw
2014-2017 Plus 15, 0% draw
2007-2010 Plus 16, -15% draw
2001-2004 Plus 38, -19% draw
2012-2015 Plus 39, +6% draw

The criteria all make sense and are based on on healthy financial metrics, good current or increasing earnings yield, and good ROC. In actuality I didn't make anything up, just stumbled on this while going through the p123 tutorial.

It picks stocks from Canadian market but excludes energy, mining and REITS
It doesn't seem to matter, because it does fine if I expand to entire Canadian stock market and even just with the TSX

The only problem is that it doesn't have the excess returns in most of the other stock indices. It seems to beat the SPY500 quite often but trails or at least matches all of the Russel indices.

So am I curve fitting, or is the TSX market fundamentally and grossly different such that what works on TSX shouldn't be expected to work in the US?
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

I started most of my screens long before I did my first backtest. Instead, I relied on the backtests of others. I had to wait until broker's fees came down and my relatively small positions made sense. For the more complicated strategies, I had to wait for free or low-cost stock screening sites.

I'm using low maintenance value screens that have worked for a long time and, hopefully, will keep working. I haven't been very tempted to try to develop high performance screens or modify those of others. Some of the people on P123 claim to be making 40 or 50%/year. Maybe some of them are even telling the truth. If I tried to do that, I don't think I'd have the confidence to keep at it after a underperforming year or two. Plus I don't want to be trading every week.

It's interesting to look at the Designer Models. These were supposed to be the best by the best. A lot of them have been deleted. So, what you see already suffers from a large amount of survivorship bias. Not many of them have outperformed.

I've found some differences between Canadian and US markets. In the US, my 6 factor value composite works better then my 5 factor value composite. In Canada, it's the reverse. And the Canadian small-cap backtests are better. Some P123 users claim their strategies work better in Canada. It's not as "efficient". They're keen to get access to European data because quant strategies work better there.

I can't give much advice on whether or not to start playing with real money. Make sure your simulation is as realistic as possible. Adequate liquidity and slippage. You said you are starting with a small amount. Have an exit strategy. What would make you abandon your model?
nisser
Veteran Contributor
Veteran Contributor
Posts: 2079
Joined: 11 Nov 2007 21:24

Re: Long-Term Backtests

Post by nisser »

Good questions!
I did some more testing over the weekend.
What reassures me more about the criteria is that if I test the bottom 30%ile, it has atrocious returns. It actually gives a negative annualized return. Similarly if I take the middle two third percentiles, it does beat the index but not by such a large margin and the drawdown returns to index-like levels.
If the criteria are purely by chance, than that correlation shouldn't exist.

But what continues to bother me is the discrepant results in other indices. It draws the S&P and only beats the Russel1000 by 2-3% which is somewhat reassuring I guess!

Having said that, I think I'll put some money in!
Market usually goes down when I buy stuff, so I've hedged it by selling some bank and pipeline stocks to stay neutral :rofl:

As to what would make me rethink?
Out of 19 years, it's only underperformed in 3 years (1999, 2000, 2015)
It's drawn even 4 years (2005, 2008, 2010, 2011)
And the maximum yearly drawdown has been 35% (2008). The drawdown every other year has been no more than 20-22%

I suppose if it were to underperform 3 years in a row or drawdown more than 25% (short of a market exploding like in 2008) then it may imply that the system isn't working any longer.
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

nisser wrote: 03 Dec 2017 16:54It draws the S&P and only beats the Russel1000 by 2-3% which is somewhat reassuring I guess!
If you're talking about the S&P 500, it should have about the same return as the R1000. Are you sure you're using total return benchmarks in both cases?

So you're going to have two screens? The Dogs and the new strategy? Should be interesting.
nisser
Veteran Contributor
Veteran Contributor
Posts: 2079
Joined: 11 Nov 2007 21:24

Re: Long-Term Backtests

Post by nisser »

Yup. I put in the orders for the screen and I'll do the Dogs one later in the week

1999-2017 (first time I did this time period, kind of nice to see it perform well!)
Screen 438.80% vs S&P 500 (SPY) 202.86%
Screen 375.29% vs Russell 1000 (IWB) 211.19%

2005-2017
Screen 185.29% vs S&P 500 (SPY) 181.62%
Screen 214.70% vs Russell 1000 (IWB) 187.36%

2007-2017
Screen 94.85% vs S&P 500 (SPY) 132.19%
Screen 152.84% vs Russell 1000 (IWB) 134.59%
DenisD
Veteran Contributor
Veteran Contributor
Posts: 4081
Joined: 19 Feb 2005 01:24
Location: Calgary

Re: Long-Term Backtests

Post by DenisD »

Here are some rolling backtests I did in November for my Canadian and US small-cap value screens. For both markets, I tested 5 and 6 factor value composites based on the work of Jim O'Shaughnessy. The number of stocks was 10 or 20. The holding period was 13 weeks or 26 weeks. In addition, I tested the best decile and the small-cap universe.

In both countries, the lowest market cap is about $100 million in the respective currencies. If we get a bear market, it would be significantly lower. In the US, the backtests bought the 10 or 20 highest momentum stocks within the approximately 120 stock best value decile. In Canada, the best decile contains only about 30 stocks. And it has been less than 20 in the past. So I tried expanding the universe and selecting from the best two value deciles. Both changes worsened performance.

The 6 factor composite worked best in the US and the 5 factor composite worked best in Canada.

The links to Excel Online spreadsheets: Canada 13 weeks, Canada 26 weeks, US 13 weeks, US 26 weeks.

Each spreadsheet contains six pivot tables with associated pivot charts. The first three average excess returns over the benchmark by year, month and week within quarter. The fourth averages excess returns by different ranges of benchmark return. The fifth averages total returns of the screens and the benchmark by year. The last averages standard deviation.

I ran some normal backtests too. I used the 6 factor composite in the US and the 5 factor composite in Canada. For each country, I started backtests in each of the first 6 months with 10 or 20 stocks held for 13 or 26 weeks. I downloaded the results and created pivot tables and charts with rolling 12, 60 and 120 month returns. Hopefully, it all came out right.

Here are the 10 year returns to the end of September:

Code: Select all

Country  Positions   Period     Jan     Feb     Mar     Apr     May     Jun Average      BM
Canada          10       13    12.5    18.1    17.3    12.5    14.4    17.3    15.4     3.9
                         26    17.5    13.6    12.3     6.1    10.9    18.0    13.1
                20       13    14.7    13.5    15.6    14.7    14.3    15.6    14.7
                         26    16.4    11.2    13.6    11.7    12.1    15.3    13.4
US              10       13     8.1    15.0     4.0     8.1    15.0     4.0     9.0     7.8
                         26     4.9    12.5    13.6    15.8     7.7     1.9     9.4
                20       13     9.6    12.3     6.9     9.6    12.3     6.9     9.6
                         26     8.2    12.1    10.9    11.9    10.1     7.2    10.1
As expected, different starting dates give very different returns.

The average US results are opposite to expectations. 20 stock screens outperform 10 stock screens. 26 week holding periods outperform 13 week holding periods. Canada is closer to expectations.

The Canadian results are much better in absolute terms and relative to the benchmark. Is that because the US market is more efficient? Or is there more of a tech bubble in the US?

The links to Excel Online spreadsheets: Canada, US.

Each spreadsheet contains two pivot tables with associated pivot charts. The first shows rolling returns of the screens and the benchmark. The second shows rolling excess returns of the screens. I've set them to show rolling 5 year returns for each starting month.

The US benchmark is the Russell 2000. The Canadian benchmark is XIC. Both are total return. Slippage was 0.35% for the screens and nothing for the best decile or small-cap universe.
nisser
Veteran Contributor
Veteran Contributor
Posts: 2079
Joined: 11 Nov 2007 21:24

Re: Long-Term Backtests

Post by nisser »

I've been playing around and trying to design systems that focus on large cap stocks in the canadian universe. I used a premise that dividend paying stocks that have good free cash flow tend to outperform others over the long term. I used a bunch of well known traditional value factors, quality, and growth factors. Moreover, I had decided to incorproate some technical factors such as low beta, low volume, average normalized range, share turnover as well as punishing companies and industries that have widespread lower earning revisions.

I used a universe that included companies with a market cap of at least a billion and currently the bucket contains 220 Canadian companies.
LC 2003-2018 13 week rebalance.png
LC 2003-2018 13 week rebalance.png (36.33 KiB) Viewed 448 times
This is the histogram showing returns over the last 15 years, with a quarterly rebalance
LC June 2013-2018 13 week rebalance.png
LC June 2013-2018 13 week rebalance.png (32.83 KiB) Viewed 448 times
And the histogram showing the same over the last 5 years.

I then did a wackload of simulations using p123 from 2010-2018 to see how this would hold up. You can't simulate a buy and hold scenario there which is what my intentions are in trying to come up with something like this. I'm looking for something where I can quarterly add to high ranked companies and hold them, and perhaps trim others if they drop tremendously in ranks. So the simulations are set up in a way in which I pick 20 stocks from various buckets (ex. Rank 80-90) and then sell them when they reach another rank (ex. Rank<20 vs Rank <70). You can see the results below. I would try to ignore the results from the simulations in the right lower corner as the negative returns there are negative largely due to slippage and trading fees (i.e. If I buy a stock at rank 30 and then are forced to sell it if it's rank is not above 90 next quarter).
The key findings are that the higher the stock is ranked at purchase, the higher the returns, no matter at which rank I sell it at, although returns seem to plateu. Buying a stock below rank 50 rarely gives a return higher than 10%, no matter when it's sold.
Large cap returns.jpg
Large cap returns.jpg (112.19 KiB) Viewed 54 times
If you're interested, I've attached the xls of the rankings drawn today.
nisser
Veteran Contributor
Veteran Contributor
Posts: 2079
Joined: 11 Nov 2007 21:24

Re: Long-Term Backtests

Post by nisser »

For some reason it didn't attach the xls. Here it is. Also it's important to note that I excluded utilities and reits and that this may not hold for financial companies. I could have excluded them but there's only a handful of them on the TSX and obviously none of the debt based/EV valuations will hold up. You could argue that 2 wrongs will make a right, and that they can still be evaluated based on other factors but I didn't look into it whatsoever.
Attachments
Large cap June 2018.xlsx
(20.97 KiB) Downloaded 16 times
User avatar
Peculiar_Investor
Administrator
Administrator
Posts: 13267
Joined: 01 Mar 2005 14:52
Location: Calgary
Contact:

Re: Long-Term Backtests

Post by Peculiar_Investor »

nisser wrote: 21 Jun 2018 13:51 For some reason it didn't attach the xls.
There is a known bug in the current version of the forum software. When a post contains multiple attachments all but the first attachment are silently dropped when the message is posted.
Imagefiniki, the Canadian financial wiki New editors wanted and welcomed, please help collaborate and improve the wiki.

Normal people… believe that if it ain’t broke, don’t fix it. Engineers believe that if it ain’t broke, it doesn’t have enough features yet. – Scott Adams
Post Reply