Lessons Nobody Wants To Learn: Enterprise Integration

It was a long time ago. I was young. It was my first job.

The job was to write a cross-compiler between two CAD systems.

On the receiving side, there was a minicomputer (a Soviet PDP-11 knockoff) running P-CAD.

On the sending side, there was a PC running some sort of a CAD supplied by, I believe, Sigmex - at least this is my recollection almost exactly 20 years later.

There was an interesting caveat to it.

You see, P-CAD is a PCB design tool.

And the Sigmex application complex was intended to create mechanical designs.

You can only imagine what sort of bribes and kickbacks could introduce a mechanical CAD system, woefully inadequate for designing PCBs, into an environment used exclusively for PCB design. I'm not even talking about the fact that there was plotting equipment designed to work with different paper size - about 2/3 of very expensive paper was being wasted into offcuts.

But I digress...

This is how ONE test run looked like:

1. Run the prototype, convert Sigmex output file into P-CAD input file;
2. Log off 6 (six) people working on P-CAD mini, busy as hell, trying to meet the production deadline;
3. Shut down the mini;
4. Take out the plotter adapter card;
5. Install the RS-232 interface card, connect the PC and the mini;
6. Boot up the mini;
7. Transfer the file from PC to mini;
8. Perform the test run on the file;
9. Withstand the scornful looks of the whole department which just lost about ten man-hours, as the following steps are conducted;
10. Shut down the mini;
11. Take out the RS-232 interface card;
12. Plug in the plotter adapter card;
13. Boot up the mini.

As you can imagine, it wasn't often that test runs were performed...

Long story short - painful attempts to produce the cross-compiler lingered for almost three years, with very little visible results. Eventually, I got sick of the task and needed to bury this dead horse once and for all.

This is what's been done:

The amount of work necessary to complete the task was doubled by adding another component to the list of deliverables. The component was a complete P-CAD file viewer.

It took less than a month to write it from scratch. Hundreds, if not thousands, of existing P-CAD files from the actual production environment were used to verify the correctness of the parser/visualizer implementation.

After this was done, the results of the dreaded cross-compiler were fed into the viewer. Since the purpose of the viewer was primarily to verify the input data, diagnostics were much, much better than "Error 0x2312980", and all the errors in the files produced were immediately obvious.

Turned out that the files were actually almost fine, with a few off-by-one and signed/unsigned conversion problems. It took a few more days to iron out those wrinkles.

The day was saved.

So, what it took to bring the project that was hopelessly beyond the budget and delivery date back in shape was a conscious decision to double the perceived amount of work to be done. Whereas it looks bizarre on the surface, it is actually not surprising.

The lessons that I learned back then were:

• You better have your APIs set in stone, thoroughly documented, with change control available, transparent and strictly followed;
• You must have a playground available for both your inputs and your outputs;
• If your input or output integration partner doesn't provide you with a playground, it is in your best interest to build one;
• Building an integration playground does NOT extend the project time and budget, on the contrary, it allows you to bring a runaway project back under control;
• If you are an API provider, it makes perfect sense for you to take responsibility for creating a sandbox of your own application, at your own expense. Complete with documentation, tutorials and help desk, of course. All this expense will pay for itself with the first API consumer, which will not bleed you with endless Q&A sessions;
• Massive amounts of test data help a lot when testing applications with either extensive protocols, and/or vague or incomplete or unknown requirement sets ("We don't know how to put this into words, but we'll know it when we see it").

It was many years later when I first saw the signs of enterprises having learned these lessons. Notable notions:

• PayPal Sandbox
• Google Checkout Sandbox
• Amazon Flexible Payment Service Sandbox is one of many sandboxes offered by Amazon Web Services
• Android Emulator is one of many examples of device emulators offered without charge by hardware and runtime manufacturers
  

But this is just about all.

Wikipedia Sandbox article was created on July 22 2006, when PayPal sandbox existed for a long time already, and Google Checkout sandbox was rolled out a month before. Two years later, it is still pretty sketchy.

The need to dedicate serious attention to sandboxes hasn't reached the attention of corporations yet.

It is counter-intuitive to invest additional resources into something you are not judged by completion of, therefore, sandboxes are not usually included into project plans.

Or, there is a reliance on "tried and true" methods of integration testing, consuming countless man-hours, extremely inconvenient and time and resource consuming because of a necessity of coordinating large numbers of people, departments and organizations, and extremely fragile due to the integration partner system downtime.

Or, simply lack of awareness.

There'll be some changes made.