Data

 Introduction 

But whatever the Corporate Strategy, insofar as most organizations are concerned, they are driven by outputs and require data to produce, support and validate them.

Robert Butler ⁽¹⁾ acknowledges that "Data elements are often defined incorrectly, stored incorrectly, not updated, and the data of interest hidden among masses of irrelevant data." He makes an armchair empiricism that "The result is that typically, 90% of an analyst's time is spent organizing data and only 10% performing analysis with it." Whilst I would not agree or deny his acknowledgement or armchair empiricism, it does bring into question the quality, meaning and utility of the analytical product that is delivered in relation to the information that is required by the customer. If the customer knows what s/he wants and can recognise it when s/he sees it, it follows that they will have shot the rapids together, hopefully without spills, and end up in a benign flow where steering is easy; everything behind is understood and the way ahead makes predictable sense.

Many organizations particularly the larger ones have vast quantities of data flowing in various directions. I have certainly seen the trend appearing whereby there is an increasing will to bring the data into a pool where it can be more readily managed; a data warehouse. I am neither an advocate particularly for data warehousing or against it. Indeed there are strong arguments both for and against their use. Two of the many opposing views are reproduced from LGI Systems⁽⁸⁾ below.

For: "To make it easier, on a regular basis, to query and report data from multiple transaction processing systems and/or from external data sources and/or from data that must be stored for query/report purposes only."

"For a long time firms that need reports with data from multiple systems have been writing data extracts and then running sort/merge logic to combine the extracted data and then running reports against the sort/merged data. In many cases this is a perfectly adequate strategy. However, if a company has large amounts of data that need to be sort/merged frequently, if data purged from transaction processing systems needs to be reported upon, and most importantly, if the data need to be "cleaned", data warehousing may be appropriate."

Against: "Data warehousing can become an exercise in data for the sake of the data."

"Organizations find that there are unlimited opportunities to add data to their data warehouse. Data warehouses, like most other complex systems, take a life of their own. Unfortunately, adding data without questioning the business value of the data can lessen the business value of the data warehouse and quickly increase the cost of maintaining the data warehouse."

Whatever your personal view, I can tell you that I have been known to make what I feel are appropriate observations on almost any subject under the sun. Donald Rumsfeld ⁽⁹⁾ famously but not quite succinctly said, "there are known knowns", etc. However, it is arguable that, at the end of the day, Mr Rumsfeld can be excused for being unclear and perhaps uncertain because he is only human after all; he is not a professional engineer or logistician. He acts out of conscience; he knows what he knows.

Conscience. What is conscience? As science means knowledge, conscience etymologically means self-knowledge . . . But the English word implies a moral standard of action in the mind as well as a consciousness of our own actions. It is also concerned with duty ⁽¹⁰⁾; we do what we think is right because we are responsible and it is our duty to be effective and efficient.

But is it enough to have a conscience? I have one I am told, but that, in itself, can only make me act out of good intent for the benefit of all; it doesn't mean that when I act out of conscience, knowing what I know, that I will always be right in all respects; in fact I could be totally wrong at times. However, if I apply my scientific knowledge properly I can be certain to some predictable extent that my expectation will be met because I am using empirical truth.

Science. As seen above, science means knowledge; it comes from the Latin "scire", to know. But science is more than knowing, "it is the systematic study of the nature and behaviour of the material and physical universe, based on observation, experimentation and measurement, and the formulation of laws to describe these facts in general terms. ⁽¹¹⁾

So where does that leave me? Well, fortunately, I am a professional engineer and logistician. Therefore if I do know something, I should act out of conscience based upon science.

 My Problem