Abstract
The emergence of e-applications is creating extremely high volumes of
new data. Indeed, the collection of information that analysts so calmly
referred to as a “sea of data” just ten years ago has now swollen to Tsunami
forces. As a result, many analysts might say that today’s data management
professionals are facing a Perfect Storm. If they don’t find calmer waters
soon, they’ll drown.
This paper is for
information technology and data management executives who (1) understand
the value of maintaining actual event records rather than their summaries,
but (2) have not yet determined a cost-effective data management strategy
for doing so with large and growing volumes of data. It introduces the
concepts of active versus supportive data management needs, high-speed
versus high-volume storage technology requirements, and micro- versus
macro-analytics.
In addition, the paper
recommends new data management infrastructure alternatives and provides
their associated cost reductions. By adopting the recommendations in this
paper, planners can strategically look for, consider, and implement informational
initiatives and concepts that, in the past, they may have immediately
discarded due to prohibitive costs.
The
Wave: Explosive Data Growth
Most analytical reports today cite exorbitant factors when projecting
enterprise data growth over the next five years. For example:
Red
Herring’s March 2000 summary, “The Age of Petabytes,” forecasted data
growth rates of 75% to 150% per year.
A
META Group analyst who spoke at a May 2000 eCRM conference projected
data increases of a hundredfold within five years through the year 2004.
Enterprises that are having difficulty coping with three terabytes (TB)
of data today need to quickly find solutions for dealing with 300 terabytes
of data tomorrow. Since the May 2000 conference, other META representatives
have validated this growth factor as well as the urgent need for in-depth
strategic data management planning.
A
recent Deutsche Banc Alex. Brown data study found that e-business data
will grow from 30% of the total data in the first year of activity (1999
in most cases) to 75% of the total data in the fourth year. This growth
represents a commanding data swell of 400% per year.
