Automating systems

[Sparky12]

Hello all

I last posted about a year ago and at the time I had been testing an Excel based system that I used to calculate my own ratings for Sat meetings in Brisbane. At the time it was showing decent enough profit over approx 200 races and I'm pleased to say that one year on and its still looking OK - nothing earth shattering but enough to keep me interested.

With the help of a few forum members I managed to get a few macros written to simplify and speed up my data capture and analysis. Essentially now I cut and paste form information from two websites and then run a couple of macros to convert my data into usable data and then to analyse the data and produce ratings. Again, nothing special here although I do apply my own weightings to different elements of form and also am able to manipulate the ratings to account for the state of the ground etc at each meeting and can also adjust barrier numbers etc when scratchings are announced. So its a bit labour intensive up front but works OK overall.

I'm really keen to test the ratings out on more meetings, esp in other States but I'm time poor and simply don't have the time to cut and paste data from websites and run the macros then adjust for going, scratchings etc.

I was wondering whether anyone has had similar issues and has managed to get a programmer to produce some software for them that replicates the (relatively simple) way in which Excel can calculate ratings and that fully automates processes that are otherwise performed in Excel via macros etc. I suppose the key issue I'm struggling with is getting the form data into Excel - perhaps there's a database that forum members use/can recommend that, with a bit of wizardry from a programmer, can be used as source data for a small programme that produces ratings automatically?

Suspect I'm up against it here! Anyway, hope this makes sense and would appreciate any help or advice.

Thanks in advance

Sparky12

[Shaun]

Hi,

Where do you currently get your form from?

What Information do you need?

What sort of time frame are you looking at to do the form?

I have used website form for the past few years and can rate a race in under 1 minute.Depending on the extra input you wish to add i am not sure what time per race we could get this to.

[Michal]

Hi Sparky,


Depends on how much you are prepared to spend. I say this with all due respect, if you are scraping websites to get free form then you are always domed to slow painfull process. Any program writen will be subject to web site changes and become useles the minute the site changes even a little bit.

My advice is purchase form ...... there are several providers and at least one is very reasonably priced. Then you have a chance to base your program on an import module, to create your own database and then automate from there onwards.

Scratchings are accesable quite legaly through NSW TAB under their licences when you are a user of their account as long as its for own personal usage, as are results that you can archive for testing.

With these tools you'll be then able to download form, update your database, run the scratchngs, run the program to automate your ratings in all states in matter of minutes.

Oh yes, ther is one further hurdle, how much money do you have to spend on a programer ? I would think that even if you use the indian IT sweat shop you will be up for thousands , many of them. On top of which you'll have to in detail explain to someone who has no idea about racing all the stuff you need.

I suppose take it one step further and automate the betting as well while you in the candy store.

Im not taking the pi..s out of you, I have done it, all except the first part I use a program to create the database and then I raid it for info but its the only part I have to complete the others are done. Took me over 2 years to get it all together, however I learned to programe and did it my self so Im not suggesting that a pro will take as long , however there isnt a clean cut line. Once the program is finnished to your specs rest asured that you'll be dishing out even more to cover the many unexpected things that pop up in the first 6 months of the programs life ...... not program errors but situations that the program was not writen for like misspeled horse name or race class and such. This will be not in the original specs and be charged extra for, you'll also not be able to use the program in the meanwhile.

If you are realy interested in doing this, learn to program your self. Buy a few books and start slowly, you already have knowlege of macrows the next step isnt as hard. It will take time but you already have a leg up on most people and me when I started you alreday have a system that works.

You will then have the ability to create your own destiny, change and adapt the program to your changing needs, NOT BE dependant on anyone or anyones program. You'll know what does what , why and where. Im sure that there are several skilled people here that can write something for you, me included, but you are still at their wimm. It takes an inteligen person to put a worth while method together, extend further and become self relient for its automatic performance.

Good luck

Michal

[Sparky12]

Shaun -to answer your question I use VirtualFormGuide for certain elements of data and Racing and Sports for remainder of data. I also use TAB for results so I can test output. I can rate a race in possibly 2 minutes - 1.5 minutes if no scratchings but although its quick I do still have to think a bit and it can take a while to update some base data (barrier stats etc) before I get going on the form.

Michal - Can you give me a heads up on some of the better places to purchase a good form database as suggested in your mail? If I had the time I'd love to do the programming as suggested.

[Chinbok]

<link rel=\"\&quot;File-List\&quot;\" href="file:///C:%5CDOCUME%7E1%5CONeillD%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C01%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:WordDocument> <w:View>Normal</w:View> <w:Zoom>0</w:Zoom> <w:Compatibility> <w:BreakWrappedTables/> <w:SnapToGridInCell/> <w:WrapTextWithPunct/> <w:UseAsianBreakRules/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><style> <!-- /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0cm; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";aa @page Section1 {size:595.3pt 841.9pt; margin:72.0pt 90.0pt 72.0pt 90.0pt; mso-header-margin:35.4pt; mso-footer-margin:35.4pt; mso-paper-source:0;aa div.Section1 {page:Section1;aa --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman";aa </style> <![endif]--> Hi Sparky,<o>
</o><o></o>
I was doing something similar using the Unitab form, Racing and Sports neural ratings and Winner in Six ratings (these may not be available now.) I used the web query function in excel to automate the process after setting the date and pressing a button. As mentioned previously, whenever the website format changes, you need to alter the code, however this doesn’t happen too often.

<o></o>You can have my excel file that contains the macros if you want to see examples of automating the web scraping.

[Shaun]

Sparky,

I would do much the same as you except i don't use any extra data other than the 3 start form and fields from VFG.It can be a bit of a challenge to extract all the form but i have got most of it worked out, they did make a change to the date format awhile ago so i had to change my sheet to get the days since last run, i will post my new sheet with form included so you can see the formulas, minus the important ratings:)

[Michal]

Hi Sparky,


Cyberhorse has form for all TAB races in AU for about $270 a year which is penuts really. The have 5 starts and these come in a zip in a html format. This should give you all or most of the relevant data to do most things with to build a database. If you wanted to speed that process up sibscribe to the 10 start for first year that way you will instantly have last 10 starts to each horse. Look there are elements like inrunning position that isnt too well covered for provincial and country, that is about the only issue the other is sectional distance for that there are others but as I said faaaar more expensive, it really depends what you need.

There are other places like AAP and southcoast and TRB that will sell form and data base but these are options that are many times more expensive and I dont know if you can export the data out. Im sure that there are others that I dont know about.
The above would be my first option if the data has all that you need. I hope that this answeres the question.

Michal

[Shaun]

Here it is, i have even included the ratings so good luck.

[Sparky12]

Many thanks for the spreadhseet Shaun, much appreciated, will review with interest. How are your ratings performing currently Shaun? I'm around 60% strike rate for my top 4 ratings at present, down on last year but still profitable backing all 4 blind (on paper anyway!). Top rated single selections are showing profit backed blind also.

One way in which I find my ratings are helping me is that by applying a standard analysis approach to the form for every single race, I believe I can spot false favourites more easily and quickly than might otherwise be the case. ie: my ratings (and yours) are an amalgamation of several form factors with a bit of judgement (via form factor weightings in my case) thrown in - the same analysis approach is repeated time after time and while very often my ratings will throw up the favourite as first or second rated, and quite often they'll throw up a complete roughy, occasionally the favourite will be 3rd or 4th top and often not in my top ratings at all. This gives me a bit of confidence that if nothing else there may be value in the prices of the other rated top rated runners and I'll bet accordingly. So far this approach has proved profitable.

Thanks also to everyone else for their comments.

[crazybadger]

Mate I'm a programmer and I have done something similiar - I defined my own rating/weighting system for various aspects of the form using TabOnline style data.

I have written a program that has recorded over 50k markets. I also made a theory testing part to the software so you can essentially say "pick my top rated only when X happens" and give it heaps of things like track condition, distance, numbers, etc.

The only thing I haven't done is made my rating system easy to change. That being said if you feel comfortable giving me the rating/weighting you use I could run probably code it up and run it over my data for you.

[Sparky12]

Crazbadger


Sorry for late reply and thanks for the offer. The weightings and form factors I use relate to several aspects of form that won't be captured (I suspect) through the TAB data inc jockey ratings, barrier stats for various distances etc. I've got Excel sorted now so I can change the weightings for each form factor and see what impact is on POT - flattening out the weightings (ie: same weighting for all form factors) at all distances below the longer distances >1800m produced the best POT which surprised me - maybe a quirk as only 250 races....I thought barrier position for starters would have been a key factor but not the case...

Out of interest what programming tools/language did you use for your analysis software?

[Michal]

Hi Sparky,

Short of you wanting to create for release software you need not to go anywhere else then excel. The reason most people write software in VB or other languages is that they are looking for security of code which sadly excel does not provide in any serious manner.

My program which calculates speed ratings for past 20 starts of each horse , import the data, processes all form analysis as well as paterns, assembles and updates the database was writen in excel, it contains about 150,000 lines of code and will do everything I need to do. This also includes a jockey database. Well a list of bout 1500 names the term database is just a scary term for a list of names :)
There are ways that you can use to get jockeys , crtachings and many other aspects , in excel, I can help, the site where you get your form has a forum and you can contact me through there on PM.

As you already mentioned that you create macros in excel then you are already familiar with the VBA enviroment it is not that far a step to create the code yourself.

I know that the question was not asked of me, but I can assure you that you will not be missing anything if you stick to Excel as your probraming platform. The best thing is that you dont need to buy anything you can just start ! It allows you to create interface so its not just a blind macro you can have buttons check boxes input boxes .... everything that you interface with in a normal program.

My advice is to start reall slow, create silly Helo world programs and other things to lear the methods then travel in baby steps so that you are not overwhelmed by the task.

Michal