System Testing
 

A quick question in relation to testing a system

I have a couple of systems with 6 or 7 rules that I am testing over a 4 year period from 2008, 09, 10 and 11.

After the test I see what the result is in dollar value.

Then I go through and individually remove a filter and see if the $$$ profit has increased. If it has I keep that filter out of the system. Basically going through each filter and is the profit goes up keeping it out.

If a filter is removed I then start at the top of the rules and re-test all over again until I have seek the best possible $$$$ profit over 4 years.

Once I have the results I then check to see if there is a profit each calendar year. If there is ups and downs each year, the system goes. If each year shows a profit I keep.

The last test is then to see if the results fair up for the period 2012. Come Dec 12 and I have another further 12 months results I will re-test all the successful systems to date.

Does this logic sound sound or am I going around in bends and following the crowd to the poorhouse.

It sounds alright to me. I don't think this would be described as "following the crowd". You're doing your own thing.

Thanks The Ocho

I would often see a nice juicy filter that I would like to apply like Win% rank or something alike but often in the long run it does not continue.

My other thought was going to apply filters only to a system if there was a negative profit each year for that filter. For example if I removed 2 year olds from my selection as they gave a negative return each year of testing I'm comfortable with this in theory. But really that is classical back filtering.

Its a personal preference of my that a system should make a profit each year.

That sounds pretty good to me, taking subsets as you are, you are highlighting any outliers that don't pop up each year.
If each individual year, you are in profit, then it's likely the system has legs.

Thanks CP. Lets keep on the freeway and bypass the poor house!

That's not a bad preference to have when betting I would have thought. Pity I can't come up with anything. :rolleyes:

Good luck.