Tickety, Tickety Tick

TickerTo develop and back-test trading algos its essential to have great gobs of historical data on hand, almost always at the tick level.  More often than not this data comes from your broker—OANDAAlpariDukasCopy, and Integral (TrueFX), to name a popular few—whether through commercial platforms like MT4 or proprietary APIs.  Alternately, one can use a generic downloader like Birt’s Tick Data Suite.

Over the years I’ve worked with more than a dozen brokers and other data providers so you think I’d have a handle on this stuff by now, but nothing could be further from the truth.  No, its an ever changing and messy landscape.   Continue reading

Handling Poison; Messages, I Mean

A poison message, for the uninformed, is a “queued” message that can’t be processed for one technical reason or another.  Doesn’t sound like much of a bother, but due to the way that queues operate, they can muck up the works if left unhandled.

Azure WebJobs does a decent job of handling poison messages out of the box. Basically, you’re given five tries (by default) to process a given message.  If, for instance, the database you’re trying to write to is unreachable or maybe the instructions embodied in an message were somehow malformed then the message would be automagically moved to a “poison” queue for further processing; but only after five failed attempts.  As to any further handling, it’d be up to you.

Continue reading

(Not!) In A Bind

“To sleep: perchance to dream…”  Laying aside Shakespeare’s morbid meaning, sleep turns out to be fairly important for programmers, and not only because of the perenial lack thereof.  Programming, at least at my level, tends to be mostly about the gestalt; something that I gather is the same for musicians.  Ask a skilled pianist to concentrate entirely on the mechanics and he’ll be sure to screw it up.  Let him forget the details, though, and the result can often seem like nothing less than magic. Continue reading