This editorial was originally published in Database Weekly.
The November 2011 edition of Popular Science magazine was devoted to data. In fact, the special issue was called “Data is Power: How Information is Driving the Future”. The focus of the issue was how we can use data to transform the world. According to the magazine, in 2010, there was about 1.2 zetabytes of data, and that by the end of 2011 there will be about 1.8 zetabytes (a zettabyte is a trillion gigabytes). That’s a lot of hard drive space. And just think of the hard drives required to back all of this data. I think I am going to invest in some hard drive company stock right away.
While these are impressive numbers, raw data is worthless. For data to be useful, it must be analyzed and interpreted. While storing data is easy, putting data to good use can be very difficult. But this is not stopping organizations from around the world from trying.
For example, Google is in the process of collecting every word published since the year 1500, about 500 billion words so far. While this data will benefit many different areas of study, it has already created an entirely new field of study called culturomics, which is the study of human behavior and cultural trends though the analysis of digitized texts, which some people are using to predict general trends about the future. It reminds me a lot of Isaac Asimov’s Foundation book series, where a science he calls psychohistory, can predict the future. What psychohistory does is to combine the knowledge of history, sociology and mathematical statistics to make general predictions about the future, which is what culturomics essentially does. It is an interesting parallel between science fiction and science fact.
Today, as DBAs, we are mostly tasked with the management of data, but I don’t think most DBAs really fully understand the power that data has. In fact, most DBAs focus on their daily tasks and leave the application of the data to others. This gets me thinking. What if over the next 10, 25, 50 years or so the role of DBA evolves from its current role as essentially caretakers of data, to a more interesting role, such as those who interpret it for the good of society. For example, instead of just managing data, DBAs could evolve to become more like the scientists who study culturomics (or psychohistory).
As technology advances, the routine tasks of the DBA may slowly go away. So is the natural evolution of the DBA to become more of an interpreter of the data, or will the administrative role of the DBA eventually fade away, just as horse and carriages gave away to automobiles? Perhaps we need to use some culturomics to find out?
So what do you think? Will the role of the DBA evolve over time (think 10 or more years in the future), or will it eventually die, a forgotten role that only existed for a few decades as technology marched on and made it obsolete? By the way, while the November issue of Popular Science focused on data, there was nary a word about the role of the DBA in the magazine. Is this a sign that DBAs are already becoming less important, or is it because the current role of the DBA is not fully understood or appreciated?
Brad M McGehee 
what you’re describing is data analysis which is often a part of a DBA’s job description. Those DBAs that are involved in BI/data mining/data analysis are very much aware of the value of the data that they’re working on. for those DBAs that concentrate on the OLTP end of the data spectrum (let’s call it – input), data analysis is obviously much less important.
Brad, I think you are onto something there. Most definitely the role of the DBA will change, and needs to change. To look at this a little further, let’s go back in time about 20 year. Yes, I was a DBA 20 years ago. In those days of course most databases that corporate America applications were using were on the mainframe. Those mainframe databases were mainly managed via scripts and highly trained DBAs.
Now fast forward 20 years. Mainframe databases still exist, but a corporation today probably has 100’s if not thousands of databases, instead of the 1, 2, 3 or 4 databases of yesteryear. The tools have also changed to be more automated. Now we can use point and click technologies to manage databases, which makes the job of a DBA’s a lot easier. Because of the new point and click management tools for databases we have seen a trend away from having highly trained DBAs, to instead having application programmers and/or system administrators managing databases. This is not necessarily a bad thing, but it does start to water down and blur the definition of what is a DBA.
As it becomes easier and easier to maintain databases, there becomes less and less need for highly trained DBA’s to maintain individual databases. As corporate America moves in the direction of less specialized staff managing databases, there may be a trend to take the highly trained DBAs and using their skills on something more challenging than the mundane point and click activities of managing individual databases. Possibly some of those new activities might be in mining our data as you have suggested.
Pingback: Something for the Weekend – SQL Server Links 16/12/11