How the cloud can add value to Big Data

Early adopters are already demonstrating how to get more from analytics via the cloud

Cloud computing and Big Data systems have undoubtedly been two of the most disruptive technology trends of the past decade. These developments have changed the way technology organisations operate and deliver value to their stakeholders.

Cloud computing has allowed enterprises to optimise IT operations by significantly reducing the need to invest in on-premise hardware and software, as well as affording businesses a new level of flexibility. Cloud adoption continues to show momentum, as the public IT cloud services market is expected to grow five times faster than the IT industry as a whole, according to IDC forecasts.

At the same time, Big Data technologies have enabled organisations to generate unprecedented value from data assets. Historically, high-volume, diverse data posed difficult challenges for enterprises that were used to working with traditional database technology. But new technical paradigms have provided ways to reduce the overhead required to get raw data into a data store, as well as drastically increasing the speed, efficiency and accessibility of processing large amounts of data.

These innovations have begun to enable actionable analysis on a variety of previously challenging data sources, including web logs, documents and machine sensors. Even so-called ‘dark data’ has been given new life through these technologies.

Big Data systems help organisations solve hard problems, but they normally require a significant up-front and ongoing IT investment, as well as dealing with the sheer amount of data in more ambitious projects.

Therefore, it makes sense that enterprises are turning to Virtual Cloudsviders who have expertise in managing and maintaining scalable and flexible computing and storage infrastructure. While on-premise data systems are by no means going away, organisations are beginning to effectively push the limits of analytics at scale by tapping into Big Data systems hosted on cloud infrastructure.

A recent survey by GigaOm Research of enterprise decision-makers reported that over a quarter of businesses have already started utilising public cloud resources for Big Data projects, and another quarter plan to do so in the near future.

While many of these early Virtual Cloudsjects involve high volumes of structured data, there are 3 key components that are already enabling extraction of value from massive, diverse data sets on cloud infrastructure:

Cloud Analytical Databases: These cloud-based services, such as Amazon RedShift, are elastic data warehouses optimised for analytics with existing Business Intelligence tools. This type of analytical database includes management and monitoring of the solution by the provider.

Hosted Hadoop Services: Hadoop clusters can also be hosted in the cloud, which avoids the need for on-premise infrastructure. Some Hadoop cloud offerings also include managed services, like job troubleshooting, software installation, testing and more.

Data Integration and Analytics: Databases designed for high-performance analytics can leverage cloud techniques like compression, column-based storage and high-speed inserts of structured data, which is ideal for complex queries and valuable business analytics.

Early adopters are already illustrating how the cloud can expand on the value proposition of Big Data, delivering elastic and cost-effective solutions for analysing data at unprecedented scale.

Sign up for our free newsletter