Extending IBM InfoSphere BigInsights with GPFS FPO and IBM Platform Symphony

Every day, we create 2.5 quintillion bytes of data—so much that 90 percent of the data in the world today has been created in the last two years alone. This data comes from everywhere: information-gathering sensors, social media sites, digital pictures and videos, transaction records and cell phone GPS signals, to name a few. This information comprises big data. Analyzing big data can produce key insights for improving the customer experience, enhancing marketing effectiveness, increas-ing operational efficiencies, reducing financial risks and more. Not every organization needs to manage for the full spectrum of big data capabilities, but opportunities to utilize new data, tech-nology and analysis techniques exist in almost every industry. The problem is that there is simply too much information in too many formats, coming from too many sources to manage it effectively with traditional tools. With a myriad of sources generating unstructured data, a greater variety of new informa-tion is now available. Each source produces data at different rates, but overall, data is arriving at greater velocities than ever before. In addition, organizations must determine the veracity, or reliability, of this new information just as they must with structured data. So how does an organization extract insight from the immense volume, variety and velocity of data in a timely and cost-effective manner? This is the core challenge posed by big data.