Understanding FLOSS quality

At FOSDEM I had the chance to sit on on a few somewhat academic discussions about FLOSS quality. It is an interesting issue for us as we have worked over the past three years to work on a type of "assemble" methodology that integrates itself into FLOSS practices, while at the same time ensures service level agreements can confidently be met in a more traditional professional services/enterprise context.A couple of thoughts really come to mind. First was a kind of strange interweaving of two different kinds of topics, that of product or component quality assurance, and secondly the issue of evaluating FLOSS projects which produce the components.

This is a rather difficult and interesting subject with some of the academics taking sabermetric like approaches to modeling projects as a way to determine their long term sustainability. I think the real difficulty for software and FLOSS is that its a two fold question one about a point in time solution, the other about the "fitness" of the ecosystem than generated it. FLOSS as a whole has shown remarkable ability to adapt and emerge solutions. But in software this is tantamount to measuring a supply chain, not validating the quality of a given product . But even more central to the point is that software is by its very nature conceptual - its effectiveness is in creating formal descriptions of possibilities about how to do things, but the definition of how something ought to be done or improved is based in innovation. I would claim that software development is the formal practice of codifying innovation. This is quite different from IT, which is a related practice of deploying the tools that software development produces to perform at a given service level. And IT in turn is different from consulting or business analysis, which again is a conceptual act and often involves the mapping of tools (as in build vs buy criteria) or the definition of a development activity in order to create a solution, or the "what" part.

Part of the problem with a product and manufacturing approach to software has been that solutions were treated as points in time as opposed to vectors. The shortcomings of product model adaptiveness have caused some very difficult imbalances. I mean how would you like to be a business with mission critical systems running in COBOL, and what does a desktop mean to someone who only understands it as a virtual construct as opposed to a model based in a bygone physical reality?

I have always interpreted the movement toward service orientation as an attempt to rectify the inadequate models of manufacturing and product development, to an information age industry. It was always clear that software valuation had little in common with physical goods. The value of software, and the its cost of ownership is related to the effectiveness of the supply chain that created it, and the consumers ability to tap into any aspect of that supply chain with relative ease. When this ability disappears, software becomes exceedingly expensive. Optimal software investment then, because conceptual understanding of the flow of information is never stagnant, is primarily based on the sustainability of its supply chain and its ability to continuously innovate and adapt to new concepts or their refinements.

So the question of quality then seems to me to be more along the line of how do you evaluate innovation supply chains? This is however very different from saying how to you evaluate innovation. The supply chain is set of economic transactions, many of them involving locally defined or domain defined protocols. Product quality assurance is one domain for example, another might be contractual mechanisms. In any case the combinatorial elements of such a supply chain seem in my mind to be so complex as to be rendered incalculable as a predictive whole.