There are obvious economies of scale associated with purchasing nontrivial technologies rather than reinventing them internally. After all, technology is expensive to create and almost free to share. However, one must consider a number of hidden disadvantages to relying on externally provided technology.
The fundamental problem with vendor relationships is that vendors have limited accountability for the purchased technology actually meeting the customer's needs. Once the check is signed, the technology is backed by a taillight guarantee (i.e. one that lasts as long as you can see the vendor's taillights).
Contracts are of limited effect in solving this problem. Do not expect relief from any court on the basis of a technology deliverable failing to satisfy promised properties. The legal system is not equipped to deal with issues of any appreciable complexity rapidly enough to have a meaningful effect.
The most effective means for maintaining a vendor's accountability is monetary leverage. The customer's leverage is typically greatest during evaluation, but ongoing leverage is also important for the following, often neglected, reasons:
| Cops lie, newspapers lie... the one thing you can count on: word on the street, that's solid. |
In my experience, the most reliable indication of a technology product's value is obtained by discussing it with two or three competent people who use it on a regular (i.e. non-evaluation) basis. If you work in the Silicon Valley and are not completely anti-social, odds are that you have friends of friends who fit this bill. The impression you get from such conversations is probably even more accurate than that resulting from your own evaluation.
A product's reputation is more indicative than that of its vendor, because even seemingly related products often have disjoint genealogy. Beware customer impressions conveyed though the vendor, or even references provided by the vendor. Beware short customer lists, but don't rely on long customer lists. Beware any form of "certification," because most forms of technology certification have more to do with factors other than technical merit.
Try before you buy. It can be surprising what even the most superficial experiments will reveal. Vendors tend to overstate the maturity of their technology. I know of more than two vendors who make a practice of advertising features that are defective well beyond the point of unusability. I even know of one vendor who ships blank data tapes to meet deadlines artificially!
If you plan to rely on a property of the technology that is suspected of being coincidental (i.e. there may exist undiscovered counterexamples, or it may be violated in future versions), then it is a good idea to require the vendor to guarantee the property. This won't necessarily prevent it from being violated, but it at least gives the vendor a chance to do the right thing. Ideally, one shouldn't rely on anything that is not guaranteed, but most likely there is no viable alternative.
Technology is by nature complex and volatile. Misinterpretations of a technology's function are commonplace. Outright failures are not rare. I have seen more projects than I care to admit fail because the customer relied blindly on the vendor to deliver as promised, or as his promises were interpreted.
| It is always the customer's responsibility to verify that the vendor's technology is performing the necessary role. This responsibility cannot be delegated back to the vendor exclusively. Unverified technologies are most likely broken. |
If you even consider purchasing technology, you should expect to devote some of your most capable personnel to review, evaluate and advise the vendor for a significant amount of time. This is a fundamental cost of outsourcing technology development. If the vendor resists this effort, then you need either to find another vendor or to expect a great deal of adversity. If you don't have resources to carry out such an effort, then you need either to revise your hiring practices or to seriously consider whether or not your organization belongs in the technology business.
Another drawback to using externally supplied technology is that vendors don't like to have customer-specific branches. In other words, if the vendor updates the product for you, then you will probably have to accept all the intervening changes made for other customers. For this reason, vendors are unlikely to accept bug reports on any release other than the most recent few.
This often operates in the customer's favor, because changes made on behalf of other customers ought to be beneficial or at least benign. On the other hand, if the you are relying on unspecified properties, if the specification has changed in violation of the previous specification, or if the changes cause specified properties to be violated, then you might run into problems.
In particular, one often finds bugs that reappear after they have been fixed. This is usually a symptom of the vendor mismanaging concurrent development . Tracking past problems and regressing each version against them is recommended. If problems do reappear, then use your ongoing leverage to insist that they are tested in the vendor's release regression.
It may be possible to compensate for such problems without requiring further changes to the updated technology. Alternatively, if leverage still exists, it may be possible to negotiate corrective action from the vendor. Otherwise, you are simply stranded.
There is a support trap that occurs frequently enough to be worth mentioning. Many vendors have a policy that support will be granted only if all information requested of the customer has been provided. This seems reasonable enough, but the trap is that the vendor can void his support obligation by simply requesting information that is prohibitive to obtain or disseminate, whether or not it has anything to do with the problem in question.
Ultimately, this is a question of ongoing leverage. Nonetheless, it is still a good idea to insist up front that the burden of proving the relevance of requested information will lie with the vendor.
Anders Johnson, last modified $Date: 2002/02/05 $