| Concepts for programming the Pyramid and purge data collector, system and simulation model |
|
Note things to do.
Please open up the simulation program and send me a screen shot of the page to show the variables it displays. As some of these will need changes.
Please keep a track of how many hours you spend doing each bit of it.
NEW PAGE SHOWING SIGNALS TO BE TESTED CLICK HERE ( added May 6th 2010)
Basic idea for initial version no1
Symbol list
Extract the excel sheet from IG index showing the UK stock list ( important to get the data showing tiered margin percentages so as to ensure high accuracy in the simulations ) In the previous simulations we had no measurement for broker margin rates, or NMS ( nominal market size ) so in the later years of the simulation it was buying billions of shares in stocks where the NMS may have only been 1000 shares!
For the new model we can estimate NMS from an algorithm taken from IG margin rates. Eg A FTSE 100 stock would have a 5% margin as no1 and no2 rate maybe 10%. A little stock could have up to 50% margin requirement.
So we need to pull out the margin rates from the sheet and load them into the program for advanced accurate calculations. ( This data will be needed for the actual signal generation part of the program and the simulator )
From the list we can get up to date information to show if a stock can be short sold, bought or just close out old positions, then variables need to be bunged into the program for filtering of trade signals.
Eg. If a stock generates a buy signal and IG excel sheet says closing only, we can just report it as EXIT SHORT POSITION ( If a position is open ).
Volume data is also important to store in the database as I have some things in mind to test out.
Count is not outputted by Yahoo or Google, but we can easily make an estimate. We will be getting delayed prices coming through, and busy stocks will have bid asks that are changing. Slow stocks will not change so often, so each price change recieved from Yahoo or Google can be interpreted as the count. ( I see this part as non urgent at this stage)
Data collection
Plan is to use Yahoo finance and Google finance or any other free providers of delayed data and historical data. For version 1 we might as well do some comparisons over about 10 symbols to see which of the above gives out the longest date range of historical data, and which one delivers the "20 minute delayed prices" in a clean and fast way. So the program can grab the historical data from Yahoo first, then go to google and compare the length of data sets. Obviously if google gives out 10 years of data and Yahoo gives only 8 years, then we can delete the yahoo data and use the Google data.
When a new symbol arrives on IG list, we want to know asap. But we dont want the pc hogged up getting the data automatically, so the grabbing of historical data from Yahoo or Google can be done in the same was as Windows update whereby the program informs us there is a new symbol on the list and asks us if we want to grab the data. A simple button to grab the data can be made when things are not busy, and a non pop up message box can show us if a database update is required. Try to avoid pop up boxes that can stop the program running, best to have a list view type instead.
As you said, sometimes Yahoo might change the symbol codes or processes and that is why we can to automatically have an alternative for the program to grab the delayed and historical data from. May as well do this before anything else, so we know it will always have somewhere to grab the data from. Then when Yahoo changes its codes, we will not have such a panic rush to alter the program. :) Good idea eh?
Data storage
I have unlimited size on my website which could be used for storing the data base, but need to discuss this with you. I guess you could right some code that uploads the whole database and any settings we want to store to the site. As this will be a slow process ( but important ) ...........thinking... We could have a local database and an online database. Talk this one over with you later.
The other point that good be useful for checking everything is working correctly, is to store the data in metastock format, this type of data can be read by tradestation in offline mode and would give me the advantage of having a fully operational database I can do test in tradestation in. I will give you a cd and send you some notes on this so you can see if its possible to execute. ( The date functions are slightly weird in Metastock format, but i believe they follow the same structure as usual ascii data does. DATE, OPEN, HIGH, LOW, CLOSE VOLUME)
Data monitoring
As we will be using free data from online sources the data comes through roughly 20mins late, so this delayed data will be used to alert the user when a stop is hit or a new trade is to be added. But there is no harm to add a 104 button ( which is not absolutely urgent, but will give the luxury of giving real time prices to the user)
|
|
Simulations
You have already done a lot of code for this which can be used with some modifications. I would like to start with a very very simple model which we can call PEROFF meaning percentage off the high or low. And for this signal we can stick to a single trade only with no pyramiding or purging.
Peroff means if a stock is falling in a down trend the lowest ( price ) is monitored ( we can use either mid, low, or close for this, and would like to have the choice of which field to use) but for this version 1 we can use (close) to keep it simple. ....so the price is falling and we hold the lowest close as a variable and if the simulation is set to 10% we then buy that stock once the close has risen 10% or more above the lowest close on record. This code will run from the very first bar of data right to the last date we set. So version 1 will have peroff buy and peroff sell variables, and we can then test all numbers from 1 to several hundred percent to find the best average to use for long and for short. ( must allow different percents for longs and shorts )
I would like to keep a good record of the dates used and the list of stocks used during all the testing. And importantly to stick to the same symbol list and data range to get exact comparisons.
Using BID-ASK averages from online data sources to make simulations more accurate
( See below section on slippage to make sense of this part)
As the online data sources spit out the bid and ask, we can make a pretty accurate guess of the bid ask spread average and store it in the database. If we take snapshots of the bid ask spreads at regular intervals through the day ( maybe 9am 1am 3am ) , we can then add the 3 together and x 0.3333 to get daily average spread. Then we take the average of those over a few days and bung it in the database ( which must permanently store it against each symbol so as to give a fixed percentage amount that always gets used in the simulations, this ensures that consistency is achieved in the testing)
Then when running through simulation we can first set the slippage percent amount in the paragraph below, and then if we have an example like the long entry with 10% slippage below in bold blue font we can take the computed 123p price and if the average spread for the stock we are simulating is 1% our new entry price for the simulation will be (123 x 0.01) + 123 = 124.23p Obviously you add it to the longs, and deduct it from the shorts.
Slippage feature
If a buy stop is at 120p and we run a simulation over it, that bar of data has a high of 150p and a close of 148p and open at 110p
The actual price with zero slippage will be 120p If we set slippage to 100% in the simulation we mark the trade entry price at the "worst price of the day" which is 150p ( when buying )
To work out the algorithm for slippage we take the stance first so in this example it will be long. ( short or flat now but about to go long )
If stance of new trade =1 then "worst price of day" = high (150p) : stop price = 120p slippage range = high - stop (30p)
entry price for 100% slippage = worst price of day (150p) entry price for 50% slippage = stop price + ( slippage range x 0.5 ) ( 30 x 0.5 ) = 15 = 120 + 15 = 135p entry price for 10% slippage = stop price + ( slippage range x 0.1 ) ( 30 x 0.1 ) = 3 = 120 + 3 = 123p
We can have a slippage input box to type in the amount of percent slippage used for the simulations. I guess we can make 15% as default value.
For short entry with an entry stop at 120p and a low at 90p ( long or flat and about to enter a short)
If stance for new trade = -1 then worst price of the day = low(90p) , stop price = 120p slippage range = stop - low ( 120-90 = 30)
entry price for 100% slippage = worst price of day ( 90p) entry price for 50% slippage = stop price - ( slippage range x 0.5 ) ( 30 x 0.5 ) = 15 = 120 - 15 = 105p entry price for 10% slippage = stop price - ( slippage range x 0.1 ) ( 30 x 0.1 ) = 3 = 120 - 3 = 117p
Then the charges imposed by IG index can be estimated over time ( by me ) and then I can figure out the correct slippage amount that is accurate, and slightly increase its harshness to make the testing very strict. This will then show the strengths and weaknesses of each signal we test out.
|
|
Pyramiding and Purging logic
Pyramiding
When we buy stage 1 of a trade, we want to risk a percentage of our max-risk on it. As it goes up we want to add more positions ( each smaller than the first one ( so as not to raise the average entry price too much ) The optimal values for how many to buy 1st 2nd 3rd, and how many percent up it has to go before adding are the critical variables I need to know.
Some people have this idea:- If you buy at 100p (risking £500 to the stop) with a stop at 90p and the stock rises up to 115p and the stop goes up to 105p you have 5p locked in, so you can calculate how much the 5p is in money terms, and then if your risk reduces down to £350 from £500 you can then add another £150 worth of risk by adding to the trade at 115p, then if it goes up to 125p and the stop goes up to 115p you can then add more risk. etc etc.
This method ensures that losers that go wrong at stage 1 will only lose £500 but winners that win big will have a much bigger trade size and this helps the risk-reward ratio to become more profitable.
Alternative methods. We can use ATRs (average true range), so if a stock rises by 1 atr from entry price we can add to it. Percentages of the yearly high and low can be used also. I will do some algorithms to test when we get to that stage.
|
|
Diagram showing various pyramiding choices. This simulation method will be able to help make a choice to ascertain which ratio is most profitable, and how many percent a trade has to move away from entry before adding stage 2 and 3 to the original position. From experience in testing model before, pyramiding makes little difference to excellent trending stocks with excellent systems attached. As trending stock are becoming rarer, it is logical to expect that pyramid entries will become more important.
Objectives of the simulation.
|
|
Purging
When a lot of stocks are held in a portfolio, the crappy performers which have not yet been stopped out can be reduced in size or closed out completely. Purging can be defined in code by using number of days and profit and trade size algorithms
EG.
Bought £1500 worth of XPP stock, had it for 50 days and profit = £30
So we take 30 and divide by 1500 to get a percentage ratio which = 0.002% and if this is made over 50 days we divide the 0.0022 by 50 days to give us a percent per day variable which is a crappy 0.0004%
The program can run through the portfolio and show which ones are the worst performing ones and either close out the trade or close out 10% of it or 50% of it depending on what we have told it to do.
This feature can be tested in simulations to find out what works best.
All this boring sounding stuff is very very important to achieving high profits and is probably more important than anything else. The end result of running it all is to end up with a portfolio full of fantastic trades at large size and all the crap gets reduced more and more as it goes wrong.
Probably in actual trading AND simulations, I will have a maximum number of trades that I want to run at one time, and purging can be a useful way of replacing crappy positions whenever a new trade comes along. I have to think a lot more about this part of it, as IG index has 1500+ symbols and I dont really want more than 100 open trades on to worry about. This problem is probably the most complex one to solve. Any ideas? Maybe just stick to stocks which have done well in past testing? Simulate over recent data to produce a table of the best recent performing stocks?
Acutal trading platform display
When we are up and running, the main trading page wants to show something similar ( or identical )to the trade screen ( which you have already made on the other program) but if its possible, this can show which "stage" the stock is at from 1 to 3 or 1 to 5. Sortable by size of trade. Highlights which fade slowly over 30 seconds or so to show when prices have changed. ( luxury non urgent feature )
Also we want to see stocks which are "about to signal" or close to stops, so i can get ready to hit it as soon as the price hits the stop.
End final result product version idea
Latest ideas for simulation and - RTS trading platform (September 2010)
We must test sub accounts with full Kelly percents. This brings both accuracy purity and precision of Kelly into the system which was absent before when we were dividing the bets up by overall Kelly then dividing again by N trades The filter required for the simulation must divide 1500 IG stocks down into the best 100 or 50 stocks. Or If using GFT, then it must divide 5000 down into 100 or 50. How do we do this?
The systems Precision stop and Donchian and Percent are still SYMMETRICAL.....this is illogical. So we must test ASYMMETRICAL models, as shorts and longs do not always have to be the same entry.
|
