To develop and back-test trading algos its essential to have great gobs of historical data on hand, almost always at the tick level.  More often than not this data comes from your broker—OANDA, Alpari, DukasCopy, and Integral (TrueFX), to name a popular few—whether through commercial platforms like MT4 or proprietary APIs.  Alternately, one can use a generic downloader like Birt’s Tick Data Suite.

Over the years I’ve worked with more than a dozen brokers and other data providers so you think I’d have a handle on this stuff by now, but nothing could be further from the truth.  No, its an ever changing and messy landscape.  

For instance, OANDA currently streams “live” ticks via their REST API but if you want historical ticks, you’re SOL; the best they have on offer is 5-second bars.  Even worse, those bars don’t get assembled from their institutional quality feed but rather through their REST API.  You’d think that would amount to pretty much the same thing, at least under the covers, but no.  Turns out the REST data is sub-sampled to only two or three ticks per second; an order of magnitude reduction compared to their institutional offering.  Even so, the data can vary depending upon the server you end up connecting to so there’s no one source of truth.

Integral does hands out ticks with consistently repeatable IDs no matter which server you hit but they only offer an institutional-grade feed with a corresponding not-so-minimal minimal balance; $50K, if memory serves although it’d be better to commit at least $250K and be an ECP if you can.  By point of comparison, I opened my most recent OANDA account with only $10,000 but they’ll let you play for as little as $500.

Alpari doesn’t offer any sort of API, which aces them out for me even though I keep on hearing good things about the company.

DukasCopy is flat out superb from both a technical and data perspective and would undoubtedly be my broker if I lived outside of the US.  Unfortunately, I’m a US citizen so I can’t trade through them.  Gotta love Dodd-Frank.

And it gets worse from there.

You might ask why I don’t simply get my data through one of the platforms, like MT4 or NinjaTrader.  They’ve been at it for years plus they’ve made a lot of sweet broker deals so you don’t necessarily have to open huge accounts to play.  Problem is, the quality and consistency of their data leave much to be desired.  Moreover, I did most of my trading through proprietary programs, so speed it critical.

To that end I’ve created a long succession of data feeds and downloaders—both FIX and proprietary—the most generic of whichI have open-sourced here for your trading pleasure.  HistDataFetch downloads free tick-data from HistData.com, arguably the best non-broker historical Forex provider extant.  You can download the VS2017 solution from GitHub.  Be sure to follow the (minimal!) setup instructions in the README.md file.  Sorry, I don’t have a pre-compiled version of the code to share with the non-programmers in the crowd but I hope you can appreciate the fact that I have no desire to foist (and support!) another trading utility on the world.

Enjoy….