|
FXCM Data, Rating of Pesavento Map |
January 3 , 2007
16:34 Ensign Howard: Welcome to today's training class with the Ensign staff...... Mike and Howard will be the trainers today.
16:34 Ensign Howard: We are excited to invite you to try the new FXCM data feed with Ensign Windows.... and today's class will show the steps you need to take. If there are several who join us in this test of the feed, and the load test goes well, then we will begin a Beta phase and iron out any observed issues before we have an official release of the FXCM feed. This has been a monumental project at Ensign to develop to the FXCM feed.
16:36 Ensign Howard: We have had various starts on the project over the past couple years, and for one reason or another those initial starts faded out. Work on the project began in earnest last June, so it has been 7 months of major effort on the part of Mike and myself to make it all happen. Kudo's are in order for Mike as he has been the primary developer on the project.
16:38 Ensign Howard: His expertise in developing servers to handle the feed has been key to the success of the project. The forex feed is from FXCM, who is a major player in the forex market in the US. They have a feed available for developers, and 90% of the developers elect to work with a simple snapshot of the symbols.
16:39 Ensign Howard: Ensign wanted to take the higher road, which was more difficult to implement, of having a live feed from FXCM with tick by tick updates, instead of snap shots. This will set Ensign apart from the majority of others who claim to have a forex feed into their software. The FXCM feed into Ensign is a tick by tick updating feed, so you are definitely real-time, and this will be higher quality and speed than a snap shot type feed which updates every few seconds.
16:41 Ensign Howard: Our implementation also has a refresh capability for tick base charts, intra-day charts, daily, weekly and monthly. Users are welcome to track the forex symbols from FXCM, even though they do not have an account with FXCM. The feed is from FXCM to the servers at Ensign Software in Salt Lake.
16:42 Ensign Howard: When you connect to the feed, you are connecting to the Ensign servers, and not to any FXCM server. This way we can have as many client connects as are wanted or needed and it is no change in the burden to FXCM. The refresh data is also maintained in a database on the Ensign servers. So if there is a fault in the data base, kindly bring it to our attention and we will see if we can find a way to correct the database.
16:44 Ensign Howard: We can refresh the data base from FXCM, but that takes several hours to accomplish. In fact, rebuilding the refresh database is a project that is ongoing as we conduct the class. However, enough of the refresh database is in place for us to be able to have a good class, and by tonight the balance of the database will be rebuilt so that it is more complete.
16:46 Ensign Howard: The coverage of symbols are 21 symbol pairs from FXCM, and the inverse relationship for these same 21 pairs. So while FXCM only has 21 symbols, the feed from Ensign will give you 42 symbols. These 42 symbols will show on the quote page in the Forex market group.
16:48 Ensign Howard: The set of 42 symbols will be automatic, meaning there is no Setup | Manager list on which you have to post the symbols to get them.
16:48 Ensign Howard: What we have done is implemented the processing of the FXCM feed to be in parallel with the data feed you subscribe to. So you can be an eSignal user or IB user and optionally elect to also have the 42 forex symbols fed by the FXCM feed.
16:50 Ensign Howard: On the Setup | Connection form you will find a frame for which Data Source you use, and below that a frame for the Forex Currencies.
16:50 Ensign Howard: You will check the FXCM to connect to the FXCM feed, but there are a couple things you must do FIRST.
16:51 Ensign Howard: You MUST download and install a program that handles traffic with the Ensign server. This program is called EDS and is our Ensign Data Server.
Since the following was in the class, Howard has made the EDS program available as a download through internet service and that is the recommended way to obtain EDS.
16:52 Ensign Howard: After you install the EDS program, you go to the Setup | Connection form and enter the Port which is 10000 and the IP address to our server which is 206.71.64.14
16:52 Ensign Howard: Now, lets do the step of helping you get the EDS program.
http://www.ensignsoftware.com/download.htm
16:53 Ensign Howard: On the Download page of the Ensign web site, we have links for programs to download. The link you click on is the 6th one on the top of the page, and it is labeled Ensign Data Server. I will update the comment that says it is for Cybertrader users..... in fact the download is for both Cybertrader and for FXCM users. So get this Ensign Data Server.
16:54 Ensign Howard: Click on the link and select the option to run the program. Install the program in the default locations, which expects your Ensign to be installed in C:\Ensign folder.
16:55 Ensign Howard: We will pause a few minutes at this point in the training to give you a chance to go download and install the EDS program. Perhaps you can use the Thumbs Up icon CTRL+T in the chat member list to indicate you are ready to continue
17:00 red: Does the symbol cad/eur use the same data as cad/eur+? I notice different values. Does the + come from the Ensign server1
7:00 Ensign Howard: The 42 symbols from the FXCM will have the + appended to the pairing so that these symbols are different than symbols from eSignal and from IB. You can have independent charts of forex currencies updating from each vendor simultaneously.
17:01 Ensign Howard: Now go to the Setup | Connection screen
17:02 Ensign Howard: Enter 10000 as the port number if that field is blank.
17:02 Ensign Howard: Enter 206.71.64.14 as the IP if that field is blank.
17:02 Ensign Howard: Possibly those fields already show the correct values.
17:03 Ensign Howard: Then put a check in the FXCM feed checkbox
17:03 Ensign Howard: You should see a brief image of the EDS program run and then minimize to the Start task bar.
17:04 Ensign Howard: Go ahead and normalize the FXCM program and if you have the Show Data box checked on this form you should see data flowing up the screen on the FXCM form and you should see symbols flowing on the Data Feed window in Ensign. (Setup/Connection)
17:06 Ensign Howard: This is what the FXCM program looks like if normalized and the Show Data box is checked.
17:06 Regent: nothing flowing onto screen
17:07 Ensign Howard: regent, on the fxcm program form, try clicking the Disconnect button and then the Connect button
17:06 Tyho: I don't see it. Is it because I installed the EDS in C:
17:07 Ensign_Mike: The installation needs to be in the same folder as Ensign. The default is C:\Ensign. If you installed elsewhere, you will need to change the installation location of the Ensign Data Server.
17:08 Tyho: well my Ensign folder is in D: Is that a problem ?
17:10 Ensign Howard: Tyho, you can install in C:\Ensign. It should work though Ensign is on D: Right now the program is looking for the FXCM to be in C:\Ensign\EDS\FXCM. So just the EDS needs to be in C:\Ensign while all of Ensign is still on D:
17:08 Ensign Howard: Another place you can check to see the feed succeeding is to click the Q button for quote page and the Forex button on the bottom. The form should be showing the 42 currency pairs
717:09 Regent: Howard, could you show the Q page so I can check to see if it's the same as mine?
17:11 Rice: program is in the correct folder. I have no quotes at all, static or otherwise
17:14 Ensign_Mike: Rice, click Setup | Connection and uncheck FXCM then recheck it again..
17:11 Ensign Howard: OK, you can open a chart from the quote page and refresh it if it lacks data. Initial refresh might be a bit longer (ie 10 seconds) because it is harvesting so much from our server. Refresh thereafter should be snappy
17:13 Ensign Howard: intra-day bars will be back to July -- daily bars will be back a few years
17:14 Ensign Howard: Ticks will be back 10,000 ticks right now, but data base will increase over the coming days to an intended target of 100,000 ticks
17:16 Ensign Howard: Now, one word of explanation is that we have force a 2 decimal shift on the Yen quotes. This makes it standard with the way we quote Yen currency from our other data feeds. The JPY/USD+ quote shows 0.83794 when the ACTUAL price is 0.0083794
17:18 Ensign Howard: I find the extras 2 leading zeros in the price unuseful and just harder to read. So we quote it as 0.83794 by multiplying the actual quote by 100. Easier on the eyes and mind and chart scale to see it this way.
17:18 taku: Howard are these symbol going to allow tick, volume and range.
17:18 Ensign Howard: yes taku, all of the above
17:20 vegasallen: Working great- love it
17:20 Ensign Howard: Forex is 24 hours, so here is example market hours page for the EUR/USD+ symbol.
17:21 Ensign Howard: Here is example 5-min chart updating tick by tick.
17:27 Ensign Howard: Database has been rebuilt so far -- for intra-day bars, back to July 2006 -- for daily, back a few years. For ticks, right now 10,000 ticks, but intend to have 100,000 refresh for each symbol
17:28 taz: hi I am getting socket error connection reset by peer
17:29 Ensign Howard: taz, Mike is looking into that, 1 moment
17:30 Ensign_Mike: taz, Normally this is due to a firewall not allowing the connection. Look at a local (software or Windows based firewall) or other (router, external firewall, etc.).
17:29 red: Forex is a strange kind of beast, does each tick in the Time&Sales window represent a sale, or a change in the bid/ask spread? The volume doesn't represent any size does it.
?:30 Ensign Howard: red, we are charting the BID. So the Last field is the Bid as it changes. This is equivalent to the IB symbol of EUR/USD-(BID)
17:30 red: ok, that helps, tks
17:30 iam: eds: no need to connect to it?
17:31 Ensign Howard: iam, when you check the FXCM feed box on Setup | Connection, EDS auto runs. Uncheck the box and EDS auto ends.
17:35 Seth: Reference Chart Properties: Under "Settings", I have A/B checked and I'm seeing a variation in the Ask/Bid window of my chart. Is the program tracking volume hitting the Bid, or lifting the Offer?
17:36 Ensign Howard: Seth, there is no volume with the tick, so we use a default volume of 1. So the volume you see is a tick count total and it is treated at the Bid on a down tick and at the Ask on an uptick.
17:37 Seth: tks
17:39 Ensign Howard: In this chart example, the data with the volume and A/B is what I collected on-line, and the flat volume ahead is from the refresh which had no volume information in the refresh bars.
17:40 Ensign Howard: Ok, we will get a video put together about the connection process and have that on our web site, hopefully yet this week. We will be in beta phase, so please be patient as we groom the whole process so it is smoother and more reliable. After a couple weeks, we should be pretty good at the FXCM feed.
17:42 iam: How do you see the FXCM symbols in the quote board? Should they automatically import?
17:43 Ensign Howard: iam, they auto show on the Forex market group page and can be transferred to any custom quote page
17:42 Ensign Howard: Let's move on to one other topic and that is I have had many questions lately about the information showing on the Pesavento Map
17:45 Ensign Howard: Here is example of the Map on the ES 2-min chart and I just recalculated the map by pressing the = key.
17:46 Ensign Howard: I have been working on a project to evaluate a 'goodness of fit' so that a user can consider whether a map is working OK, poorly, or very well, or possibly needs to be inverted. So to that end, I have worked to quantify things that are part of the beauty you see and would consider a Map to be a good fit with the chart.
17:48 Ensign Howard: The map on this chart is a very good fit, and so to indicate that there is a little rating system which can optionally be shown on the top of the chart in the message.
17:51 Ensign Howard: On the chart example, the rating is shown at 19.0. The 19.0 is really two separate ratings with a decimal between. It is not a decimal number but rather, the fit for the map in the normal position is a 19 and the fit for the map in the inverted position is a 0.
17:52 Ensign Howard: So the number on the left side of the decimal is the fit for the line as drawn, and the number to the right of the decimal is a rating for the line drawn in the inverted position. If you press the I key to invert the map the values will show as 0.19
17:53 Ensign Howard: My design of the rating is using a pivot of 10 as the transition between something that looks pretty good and something that looks kind of poor.
17:53 Ensign Howard: So a value that is 10 or higher is a good fit, and a value that is below 10 is a poor fit. The higher the number the more excellent the fit. The range of values seen have been as high as 24, but that is rare.... so it is best to think the range will be 10..20 for good fits - and 0..9 for poor fits.17:55 Ensign Howard: Now the 19 in the example above had the benefit of hindsight as the fit was recalculated through the last bar on the chart. What you want to do next is slide the chart leftward so that more bars are added to the chart but do not recalculate the map. The fit values will change, and if you start a new day, the rating is FOR THE DAY ON THE right most part of the chart, though the chart and the map might be showing more than one day
17:57 Ensign Howard: In this example, chart was moved leftward to show some bars for Dec 29th. The 14.0 is for the bars on Dec 29th, and NOT for anything on the 28th. The fit is still considered to be very good as both the map and the actual are in an uptrend.
17:58 Ensign Howard: In this updated view with more bars, the rating is 13.7 where the normal position is still superior to the inverted position since 13 is higher than 7. The map and the bars had a bit of a divergence at that top turn, and that keeps the rating from being a higher value. Yet, the 13 is still considered to be a worthy fit
17:59 Ensign Howard: The actual and the map have had more divergence and the rating has dropped to a 12 and if plotted inverted the rating would be a 10
18:00 Ensign Howard: The map was going lower and the actual finally also went lower, and the rating climbed back to a 13, and the inverted position is a worse fit than before and dropped to a 5.
18:01 Ensign Howard: Futurestrader asked how we analyze goodness of fit, and it is a complicated thing to quantify things you eye sees and considers to be good. So what ever I have devised should be considered comparisons of relativeness. My method involves an integration of the area between the actual and the map, as well as correlation for suggested direction, as well as alignment of turning points. All these things are considered, weighted, and summed to give a final score.
18:03 Ensign Howard: Regression analysis just did not fulfill the need.
18:04 Ensign Howard: Least squares just does not work because the difference between some areas of divergence is amplified by squaring and it was extremely difficult to arrive at a rating that was useful. Oft times with least squares a score would be really bad, yet in the eye of the beholder you would say it was a very good forecast.
18:04 trendtrader: What does the secret code s9d7t0 mean?
18:05 Ensign Howard: The s9d7t0 is just some intermediate information I am returning to myself as I worked on developing and improving the rating or scoring. So these are some intermediate values for my development and evaluation. Eventually when I am satisfied I can not improve on the effort any more, my intermediate feedback will be removed and you will have just the two scores or ratings, one for the chart as shown and one for if the map were inverted.
18:07 Ensign Howard: A key use of the 2 values would be to help in the decision whether to invert the map or just plain recalculate. I have noticed that the best fitting maps are when there is good spread between the two ratings.
18:08 Ensign Howard: A 19.0 has a spread of 19-0 = 19 which is very high. A rating of 12.10 has a spread of only 2 and the fit is accordingly more bland. So the spread between them might be an indication in itself.
18:09 Ensign Howard: On sideways narrow range days, the spreads will be smaller, indicating it does make much difference if you use the map or the inverted map because both work similarly..... it was a choppy sideways day
18:09 virt5: s= spread, d = direction, t = turning point ???
18:10 Ensign Howard: good guessing virt
18:10 virt5: lol
18:11 Ensign Howard: well... thanks for attending class.
18:11 FuturesTrader: Interesting method. Thank you.
18:11 Ensign Howard: in the meantime, I expect the FXCM feed to be working sufficiently to be useful and used. We will try to keep the server on-line, and if it is off-line, it might mean we are replacing software or hardware or have had a crash we need to debug, possibly triggered by the load it was handling. We think our server design will handle 5000 clients just fine, but proof will be in the actual doing it.
18:13 virt5: What determines the "speed" of refresh...ie between asking for a refresh and getting the refresh? Seemed very slow today. Esignal....12/29 version
18:14 Ensign Howard: ok, do not know as I worked all day on FXCM feed and did not have esignal running.
18:14 virt5: Also a weekly refresh on a weekly chart did not work...had to change symbol to get a refresh
18:14 Ensign Howard: Might have been internet bottlenecks or issues with esignal server farm.
18:15 Ensign Howard: you can go to setup | Connection and uncheck the Tick or the History check boxes. That will force Data Manager to get a new connection with esignal server farm and possibly you will get a different server path and have improved times
18:16 Ensign Howard: If you ever need to see the servers you are connected to, click the Show Log button on the Setup | Connection form. In the debug window that shows you will see information like H15 which is the name of the particular server used for refreshing the ticks and intra-day bars.
18:17 virt5: ty
18:17 Ensign Howard: The History server is used for daily, weekly, and monthly refreshes and reports its server as well.
18:18 Ensign Howard: ok, thanks for attending class..... look for new upgrades every day as we will probably be releasing them as we groom FXCM issues
18:18 Ensign Howard: When you provide feedback by e-mail, try to give all needed details such as the symbol, chart time frame, quote values, pictures, version you are using, etc. I will try to make the same observation and rely on your details on where to look and what to look for.
18:19 Ensign Howard: Have a good evening....bye
18:19 FuturesTrader: G-night Howard.
18:20 Tyho: Thanx great class
18:20 Ensign Howard: welcome - bye