Not Invented Here

Like most programmers, I frequently end up thinking that—given enough time and money—I could do it better than the “big boys.”  The industry even has a standard term for this sort of insanity: “Not Invented Here!” Like many such delusions, it’s varied in the execution but very simple to explain. Just take any piece of commercial-grade software that you consider to be inadequate or flawed, then imagine for a second that you could (and should!) rewrite it from scratch; only better! As delusions go, NIH would be counted harmless enough if it only went that far. I mean who amongst us hasn’t imagined that we could be a better actor than {?????}, or a better leader than {?????}, or even smarter or prettier than {?????}.  Or, or, or…

The problem with programmers, though, is that our entire world is built on this stuff. Plasticity sorta comes with the territory, and in most cases that’s a good thing. Software can be improved and damn it if we’re not the guys to go ahead and do it. NIH is always lurking, though, and I’m writing you today to talk about my latest personal encounter with the beast.

Before I go onto my own story, though, I’ll relate a random—but by no means unusual—instance of NIH that I encountered on Wednesday.  At the meeting (which was excellent, BTW!), I chanced to overhear two otherwise reputable-looking gentleman discussing a complete re-write of Cerebrata’s popular Cloud Storage Studio. It seems that they fervently disliked the non-traditional UI and thought they could do better.  Mind you, CSS is a serious product that undoubtedly represents thousands of development hours. Even so, from the small snatch of conversation that I caught, they seemed to imply that it was a totally doable and reasonable thing to do.  Classic NIH!

My most recent personal case arose while trotting down that otherwise slippery slope: shareware.  I have a long and storied history with such things, starting with the poorly named but exceptionally profitable eBase/Mailer; an early standout in the emerging field.  Over the years I’ve published a variety of downloadable products; to varying success.  Regardless, it’s been more than a decade so it would be fair to say—on some level—that I’m, for all intents and purposes, a noob once again.  Therefore, wanting to do everything right, I did a bunch of research, some of which, I should add, turned out to be less encouraging than I had hoped.

I won’t go into my findings except to say that I came across Patrick McKenzie’s excellent “Why I’m Done Making Desktop Applications” article which, among other things made a total case for never selling a desktop app when you could sell something on the web instead.  The article was written in 2009, so it’s a bit out of date.  Even so, it was very helpful, and remains totally relevant to this day.

Anyway, the thing that started me down the NIH path was the fact that I remained resolved to create a desktop app.  I know, given all of the great advice against, it sounds a bit like hubris.  Believe me when I say that I gave it a long and hard think before deciding to proceed.  The thing that ultimately turned the tide was that I saw a clear yet under-addressed market need that looked to turn a great profit.  Running the numbers, though, it became apparent that I could only do so if my conversion rate (i.e. the percentage of trier’s that I could turn into buyers) was high; at least 2.5%.

As luck would have it, the shareware sales process is something that I’m very well versed in.  Indeed, with my partner Kenneth Weiss, I co-invented MoneyPump; a groundbreaking application-embedded shareware sales system that—and you have to trust me on this—I have yet to see equaled in a dozen years.  More to the point, the average MoneyPump conversion rate, across all of our vendors was 4.7%, so 2.5% seems doable.

Now MoneyPump was a very successful product in its day.  To blow your mind a bit, though, I successfully replicated something like 40% of the original functionality in only five days.  40% may not sound like a lot, but it addresses most of my present-day need.  To put this in perspective, the original MoneyPump represented nearly two man-years of custom Delphi development.  In a modern context, though, much of the work was wasted on custom plumbing, persistence, distribution, reporting and UI; five things that are a collective slam dunk in a modern context.

One thing that I didn’t have a good answer for, though, was merchant connectivity.  I spent three hours looking into a bunch of web-based solutions, but every one of them seemed to come up short in one or more ways.  In the process, though, I came across the new PayPal REST SDK; which was exactly what I was looking for.  The entire reason I wanted to go the in-built custom-software route is a bit too involved to get into here, but it basically comes down to the fact that by employing the MoneyPump methodology, we were able to increase conversions rates considerably when compared to more typical web-based solutions.  Given the economic parameters of the project, I was sure that such improvement would be essential.

Two hours later, I’d signed up with PayPal, downloaded the C# API and, most importantly, got it working.  Let me give the developers all due kudos.  They did a great job, both from a coding and documentation perspective.  Unfortunately, they still fell a bit short, short enough that I decided to rewrite the library.  Here are some of my reasons:

  • The code had all of these weird dependencies and architectural miscues
    • It employed the decrepit and app.config tied ConfigManager
    • It embedded log4net, deep down in the code
    • No separation of concerns or IOC; not even interfaces
  • Their library was way too big and complicated
    • I didn’t need most of the functionality, like Refunds, Stored Cards, etc.
  • They inappropriately wrapped System.Exception in PayPalException
    • Even worse, the code unnecessarily ate a bunch of standard errors
  • Their library didn’t play nice in a multi-threaded environment
    • Given their web orientation, this was somewhat forgivable
    • Regardless, I needed a library that was well-suited for the desktop
  • The REST service  includes a pretty comprehensive error reporting system
    • Regardless, the SDK ignored it completely
  • The C# code had too much of that Java stink about it
    • Java is great and all that, but un-muddied C# is a better way to go

All in all, it turned out to be a compelling list.  The good news is that I was able to do a complete the rewrite and finish my initial round of testing in only nine hours.  I kept the POCOs but redid much of the other code.  Now, I was lucky in that the whole thing didn’t turn out to be a week-long detour.  Could I have gotten the standard library to work?  Maybe. . . well sure, but never like I’d want.  On the other hand, I am well aware that none of this stuff comes for free.  For starters, I now have another significant library to support and own.  When PayPal makes their inevitable changes I’ll have to do likewise.  Also, by definition, I’ll need to do a whole lot of testing that I wouldn’t have needed to otherwise bother with.  My full test plan will undoubtedly take 15 hours or more; more than doubling my original “development” effort.  Finally, PayPal doesn’t offer a certification suite, so my library will only end up being as good as “I” think it to be.  There might be some obscure or nuanced stuff in the API that I’m totally not getting.

So, what’s the conclusion?  I wish I could say that NIH is uniformly bad.  I’ve certainly spent entire months on similar efforts, quite often to no permanent effect.  In this case, though, a custom library did indeed turn out to be the best way to go.  Regardless, the entire thing makes me more than a little queasy.  In an ideal world, I would never ever be compelled to do this stuff.  Then again, it’s possible that I’m only making excuses.  Classic NIH!

As to the code, I’ve deployed all of the source to  I’m not quite ready to vouch for the efficacy of the SquidEyes.PayPal library just yet, but as I continue to build it out and test, I’ll update the repository to match.  If things go to schedule, my library will be in a commercial product within six weeks or so.  Wish me luck.

One more thing, I briefly mentioned the Users’ Group but I want to go into the entire Users’ Group vibe a bit more.  As I said, they’re great, and the most recent meeting was extremely helpful with regard to my PayPal project.  In particular, the first speaker—John Peterson, of Neudesic—turned us on to a bunch of new features in the new Visual Studio 2012 Update 2.  My favorite was “Paste JSON as Classes.”  For those of you who don’t attend users groups, my only comment is: “Are you nuts!”  Having attended and/or hosted hundreds of such sessions over the years, I can unequivocally state that most of the cool stuff I know comes from these events.  Case in point, John saved me two whole hours on the PayPal project alone.  Assuming I continue to work with JSON, his one little bon mot might add up to hundreds of cumulative hours saved.  Maybe in a year or so, it’ll even be enough to cancel out all of the effort spent on SquidEyes.PayPal.

Can anyone say “Not Invented Here?”


One thought on “Not Invented Here

  1. Pingback: URL Hunting (Sniffing, Spelunking, Parsing…) | Squid Eyes

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s