Data storage is free, right? That’s a common view that prompted Slate technology columnist Farhad Manjoo to write in April, “…if you’re paying anyone for online storage, you’re a chump.” And in fact, consumers may distribute their photos and videos across multiple cloud services, keeping their usage of each below the threshold that requires payment. But as I noted in July (“Maintaining Flexibility and Data Security”), “… few businesses would want critical documents, spreadsheets, presentations, and applications scattered across multiple clouds.”
Nevertheless, business data-storage requirements continue to explode—and that’s particularly true with respect to test-data acquisition. Indeed, making sure to acquire and store sufficient data can be critical to the success of a project. At an Oct. 24 presentation at the Automotive Testing Expo in Novi, MI, Jordan Van Baren, domestic sales manager at Vibration Research, addressed this issue with respect to automotive road tests. He emphasized, “Sample at a rate high enough to get all pertinent vibration data. You can always filter and down-sample later. You cannot up-sample later.”
But despite the illusion of “free” data storage that the prevalence of services from companies like Dropbox, Box.com, SugarSync, Microsoft, or Amazon might suggest, data storage does have costs. That point was emphasized by G. Sammer, global product manager for AVL in Austria, in an Oct. 24 Automotive Testing Expo presentation titled “Handling the Data Explosion from Gigabytes to Petabytes.”
Sammer noted that in the automotive testing space, exponential increases in powertrain complexity and diversity are leading to corresponding increases in data volume, and data is expensive. Increasing volumes of metadata, the need to manage heterogeneous data, and the need to ensure the quality of data, test beds, and applications are all cost drivers, he added.
IT staffs and data-storage infrastructure will struggle to keep pace with the increasing data volumes, he said. One way to control storage, according to Sammer, is to delete data, although that carries risks of deleting something important. Users can add storage, but that imposes costs related to investment in infrastructure, and an increase in data volume imposes costs with respect to data mining.
Sammer advocated a seamless integration of data formats from office simulations to road tests with a common data backbone. He recommended ways of rejecting data that’s not usable, consistent, or plausible (perhaps because it was acquired by a faulty sensor) and that’s not productive (temperature sampled at 100 Hz, for example). He cited AVL’s Santorin Host as a system that can help manage data.
Van Baren at Vibration Research did not address data-storage costs directly, but he clearly is aware of the problems an abundance of data can cause. Once you’ve acquired test data in the field, he said, you can edit out dead spaces. Then, he said, you can play back your recorded file 1:1 on your shaker system in the lab, but that’s a time-consuming process and doesn’t constitute a randomized test of what might happen. As alternatives, he suggested, you could create a test based on random PSD (power spectral density) or fatigue damage spectrum.
Sammer of AVL cited predictions that data will grow by a factor of eight over the next five years, and that in 2011 only 20% of enterprises applied the information lifecycle strategies that will be necessary to deal with the data explosion. He concluded by saying that responsibility for efficient data handling lies with test engineers acquiring the data as well as IT departments.
Executive Editor
Visit my blog: http://bit/ly/N8rmKm