29 Apr The Holy Grail of System Testing (That Nobody Likes)
The Holy Grail of System Testing (That Nobody Likes)
Apr. 29, 2024
Subscribe to my YouTube Channel HERE.
There’s nothing better than having a robot do all the trading.
Discretionary trading is great. Automated trading is greater.
But the big question is: How do I build my own system?
Uh oh.
Now all the problems come out.
Do I have to only use Out of Sample (OOS) data? If so, how much?Â
But that’s just the beginning.
Do I have to use Walk-Forward testing? If so, how much data do I use out of sample? And how often do I re-optimize? And which metric do I sort by? And do I take the top result? And how much data do I use in sample? And what’s a good result? What if it loses in my in sample period? And what if I don’t have decades of data? And what about Monte Carlo testing? And on and on…
It’s daunting.
How can we get around this?
Buy something from someone you trust.
In the real world, of course we’d do that. We go to restaurants where people have already been and loved it. Same for movies. And new cars. And new jobs.
But in the trading world, everyone’s trying to be a hustling marketer, so it’s very hard to find anyone or anything that works. There are a few people out there who talk about something in a YouTube video or Newsletter, and it actually works that way on your computer.
It’s a very short list, though. Very short.
If you find a system that’s already worked for someone else, though–boom, you’re practically done. If you understand the system, you’re probably good for a long while.
Why?
Because buying a system that’s already working is using the Holy Grail of Testing.
Which is: watch something trade for six months or more.
Here’s the truth about testing. You can actually do it any way you like.
Use a lot of OOS data, or a little. Do walk-forward testing, or don’t.
Use Net Profit as your qualifier, or use the Return to Drawdown ratio.
In short, test it the way that makes sense to you. Use your brain and run some data.
And then use the Holy Grail.
Watch that system trade for six months or more.
If whatever system you’re looking at works for six months to a year, it’s probably a viable system. Keep in mind, a system can lose in the “Evaluation Period” and still be ready to go.
If the Evaluation looks just like the previous data–drawdowns, too–then it’s almost certainly a viable methodology.
That’s why a previously-made system works. It’s already passed this test.
So, we can test an idea we have, put it in our Evaluation Workspace, and watch it trade for many months. Or we can buy a system and do the same thing.
Either way, the Evaluation Period truly is the Holy Grail.
The only problem is: nobody wants to do it.
That’s okay. I’ll do it.
And in our next Newsletter, I’ll go over a system that I’ve been watching trade for two years.
Talk to you soon.
Join the free Newsletter list here .
Get Robot information HERE.
Visit my Performance Page HERE.
Buy my new book HERE.
The Inevitability of Becoming Rich is HERE.
My latest YouTube video is HERE.
Disclaimer:
It should not be assumed that the methods, techniques, or indicators presented in these products will be profitable or that they will not result in losses. Past results are not necessarily indicative of future results. Examples presented on these sites are for educational purposes only. These set-ups are not solicitations of any order to buy or sell. The authors, the publisher, and all affiliates assume no responsibility for your trading results. There is a high degree of risk in trading.
HYPOTHETICAL OR SIMULATED PERFORMANCE RESULTS HAVE CERTAIN INHERENT LIMITATIONS. UNLIKE AN ACTUAL PERFORMANCE RECORD, SIMULATED RESULTS DO NOT REPRESENT ACTUAL TRADING. ALSO, SINCE THE TRADES HAVE NOT ACTUALLY BEEN EXECUTED, THE RESULTS MAY HAVE UNDER- OR OVER-COMPENSATED FOR THE IMPACT, IF ANY, OF CERTAIN MARKET FACTORS, SUCH AS LACK OF LIQUIDITY. SIMULATED TRADING PROGRAMS IN GENERAL ARE ALSO SUBJECT TO THE FACT THAT THEY ARE DESIGNED WITH THE BENEFIT OF HINDSIGHT. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFITS OR LOSSES SIMILAR TO THOSE SHOWN.