Big Data

From Wikibon

Revision as of 18:22, 27 August 2012

Big Data Research and Analysis from The Wikibon Project

The best of Wikibon's BigData research and analysis.

Big Data is emerging from the realms of data science projects to help companies understand exactly, make decisions, and act in real-time to better serve their customers and target markets. The IT techniques and tools to execute big data processing are new, very important and exciting.

Big Data is the new definitive source of competitive advantage across all industries. Enterprises and technology vendors that dismiss Big Data as a passing fad do so at their peril and, in our opinion, will soon find themselves struggling to keep up with more foreword-thinking rivals. For those organizations that understand and embrace the new reality of Big Data, the possibilities for new innovation, improved agility, and increased profitability are nearly endless.

The Big Data market is on the verge of a rapid growth spurt that will see it top the $50 billion mark worldwide within the next five years. The analysis in this article highlights Wikibon’s five-year forecast for the Big Data market as a whole

Consider the apocryphal saying, “Nobody ever got fired for buying IBM,” but replace IBM with Hadoop. Sounds almost fanciful, no? But making that statement a truism is now the mission of every Hadoop distribution vendor.

Wikibon research in defining big data, to differentiate big data projects from traditional data warehousing projects and to look at the technical requirements. This Wikibon article looks at the business case for big data projects and compares them with traditional data warehouse approaches.

The Big Data ecosystem is exploding with slews of exciting start-ups. Below is a list of start-up vendors and the amount of venture capital each has raised, including lead investors. This list represents many of the top Big Data start-ups and Wikibon encourages community members to update this table with relevant start-ups and associated funding that are not already represented.

Additional Wikibon Big Data Research and Analysis

• Hardware Efficiency Important to Overall Hadoop ROI
• Accumulo: Why the World Needs Another NoSQL Database
• Google and VMware Provide Virtualization of Hadoop and Big Data
• Clearing Up Some Confusion on the Hadoop Wars
• Google Playing the Big Data Long Game
• The Emerging Big Data Vendor Ecosystem
• Ask More of Your Data, More Often
• The Big List of Big Data Infographics
• Google Playing the Big Data Long Game
• Big Data in 2012: Hadoop, Big Data Apps, Data Science Tools, Cloud Collision and More
• Predictions 2012: Big Data, Flash, Cloud and Facebook
• The Biggest Big Data Stories of 2011
• Definitive List of Big Data Statistics
• Navigating the Big Data Vendor Landscape
• Hadoop, Big Data Focus Shifting To Analytics and Visualization
• IBM Outperforms Even Apple: Smarter Planet, Big Data and Cloud Power the Future
• Data, VDI and Virtualization: 2011 IT Roadmaps
• Is Big Data Utopia Near at Hand?
• Ten “Big Data” Realities and What They Mean to You
• Strata Conference: The Big Data Wave is Coming
• Little Data, Big Data, Flash and Cloud Tiering: Predictions 2011

Big Data Peer Incites from The Wikibon Project

Where exactly does the value in Big Data lie? In a recent Wikibon Peer Incite call, Relay Technology Management’s Dr. Brigham Hyde zeroed in on the answer. Deriving value from Big Data, said Hyde, requires unifying unstructured content & data with structured data in a way that allows end-users to gain deeper insights than possible with analysis of structured data alone.

The Wikibon Community convened to consider whether the advent of Big Data with its focus on multiple kinds of data including unstructured data-types has turned the data warehouse into a dinosaur. The center piece of the discussion was a recent Wikibon study of the advantages and disadvantages of the Oracle Exadata IT appliance and similar offerings expected from IBM and HP, and a survey of early big data users.

Big Data Infographics from The Wikibon Project

Big Data can be a beast. Data volumes are growing exponentially. The types of data being created are likewise proliferating. And the speed at which data is being created – and the need to analyze it in near real-time to derive value from it – is increasing with each passing hour.

While the concept of data science has been around for decades, the notion of a data scientist has become an in-demand career leading to a rise of a new generation of data scientists. Social media platforms such as Facebook depend on data science to create innovative, interactive features that encourage users to get interested and stay that way. Here is a visual walk-through of this concept and the role of the data scientist.

What’s critical to realize is that 35% more digital information is created today than the capacity exists to store it; and this number will jump to over 60% over the next several years. Most of these gigabytes of data will pass through the servers, network, or routers of an enterprise, which becomes responsible at that moment for managing that content, protecting user privacy, watching over account information, and protecting copyright.

Big Data Video from theCUBE and SiliconANGLE.tv

Boston is quickly emerging as the Hub of the Big Data Universe, and we've got the goods to back up the claim. Wikibon's Dave Vellante, Jeff Kelly and Stu Miniman recently paid a visit to Atlas HQ to talk with start-ups, VCs and Cowen and Company's Peter Goldmacher about the burgeoning Boston Big Data scene, all captured live inside theCUBE.
Watch the full video here.

O’Reilly Media Founder and CEO Tim O’Reilly goes live inside theCUBE with Wikibon’s Dave Vellante and SiliconANGLE’s John Furrier at the Strata Conference 2012 to discuss the developing Big Data ecosystem, the role of data in society and more.
Watch the full video on YouTube

Hortonworks’ goal is for Hadoop to be an enterprise-grade Big Data platform and keeping the framework open source is the best way to do that, saidHortonworks CEO Rob Bearden. Speaking live inside theCUBE at Hadoop Summit 2012, Bearden also sizes up the Hadoop competition and talks about the future of the company.
Watch the full video here

Becoming a top notch Data Scientists requires a number of qualities, according to Kaggle President and Chief Scientist Jeremy Howard. Howard, a first-class Data Scientist in his own right, said tenacity is a key trait to reach the top of the profession, speaking live inside theCUBE at Strata Conference.
Watch the full video here.

Companies are starting to differentiate themselves via new and innovative analytics and applications built on top of Hadoop, said Jeff Hammerbacher, Cloudera Co-Founder and Chief Scientist. Hammerbacher also discusses how he coined the term and the role of Data Scientist live inside theCUBE from Hadoop World 2011.
Watch the full video here.

Netflix Director of Architecture Adrian Cockcroft joins John Furrier live inside theCUBE to discuss how NoSQL technologies support the company’s streaming movie business. Netflix recently completed its migration to Cassandra deployed on AWS.
Watch the full video here

Hilary Mason, Chief Scientist of Bit.ly, goes live inside theCUBE at Strata Conference 2011 to chat with John Furrier of SiliconAngle and Dave Vellante of Wikibon about the opportunities of leveraging Big Data for analysis to create new products. She also discusses the evolving role of the Data Scientist.
Watch the full interview here.

EMC President and Chief Operating Officer Pat Gelsinger goes live inside theCUBE from EMC World 2012 in Las Vegas. Gelsinger speaks with Wikibon's Dave Vellante and SiliconANGLE's John Furrier about Big Data, Cloud, and EMC's MegaLaunch II. "In the technology industry .. If you're not leading, if you're not moving ahead, you're falling behind," Gelsinger said.
Watch the full video here

Nathan Marz, lead engineer at Twitter, goes live inside theCUBE at Strata Conference with Wikibon’s Dave Vellante and SiliconANGLE’s John Furrier to discuss Twitter’s use of both Hadoop, for historical Big Data analysis, and Storm, which Marz authored, for real-time streaming Big Data analysis.
Watch the full video here.

Because it was built from scratch, Vertica’s columnar, MPP data warehouse is more flexible than competing offerings based on the open source, relational database Postgres, said Colin Mahoney, VP of Products and Business Development at Vertica, now an HP company. Mahony, now Vice President and General Manager of HP Vertica, spoke live from HP Discover 2011.
Watch the full interview here.

Hortonworks' Arun Murthy goes inside theCUBE at Hadoop World to discuss the improvements in Next Generation MapReduce, part of Hadoop-0.23. Results are more flexible data processing capabilities.
Watch the full video here from Hadoop World 2011.
Also watch Arun's video from Hadoop Summit 2012.

Jonathan Ellis, Co-Founder and CTO of DataStax and leader of the Apache Cassandra project, goes live inside theCUBE to start a great day of coverage at Cassandra Summit 2012. Ellis discusses the state of the open source Cassandra project, the dynamics of the ecosystem and the role Cassandra plays supporting real-time transactional Big Data applications in the enterprise.
Watch the full video here

The next two years are going to be transformative as to how enterprises collect and use data, said Cloudera CEO Mike Olson at Strata 2012. Speaking live inside theCUBE with Wikibon’s Dave Vellante and SiliconANGLE’s John Furrier, Olson also talked about the real-world business problems Cloudera’s customers are solving today with the help of Hadoop and complimentary Big Data tools.
Watch the full video here.

Hadoop creator and Cloudera Chief Architect Doug Cutting goes live inside theCUBE to discuss the evolution of his brain child. He’s joined by Abhi Mehta, founder of Big-Data-as-a-Service provider Tresata, who talks about Hadoop in financial services.
Watch the full video here.

Andrei Dragomir, Software Engineer at Adobe Systems, goes live inside theCUBE to talk about how Adobe is using Hadoop to improve its relationship with customers. Speaking with Wikibon’s Jeff Kelly, he also shares some advice to would-be Big Data practitioners.
Watch the full video here

Big Data Presentations and Event Archive from The Wikibon Project and SiliconANGLE.tv

A look at what Big Data is, the market opportunity, some customer use cases and how users should think about taking advantage of the opportunities. Especially targeted for explaining to an audience of infrastructure practitioners that the design requirements are different than traditional data center technologies.
View or download the full presentation here.

Big Data holds the potential to dramatically change the way enterprises do business across industries. Yet despite the enormous potential of Big Data, adoption among mainstream enterprises is relatively low. Find out what it will take to democratize Big Data.
View or download the full presentation here.

• Cassandra Summit 2012
• Hadoop Summit 2012
• HBaseCon 2012
• EMC World and Data Science Summit 2012
• O'Reilly Strata Conference: Making Data Work 2012
• O'Reilly Strata Conference: Making Data Work 2011
• Hadoop World 2011
• Hadoop World 2010

Big Data News from SiliconANGLE

• How Salesforce.com Deals With Big Data
• Why “Science” is an Important Word in the World of Big Data
• Big Data Visualization: Very Cool But Still Very Complex
• How Google, IBM and Yahoo Use Big Data to Predict the Oscar Winners
• Thinking Big About Big Data
• A Perfect Match: When Big Data Met Node.js
• Survey: For Big Data, the Large Enterprise is Turning to Linux
• 5 Big Data Tools Built On Hadoop
• How Should the CIO Develop a Big Data System?
• What is Big Data? 4 Definitions

Additional Links and Resources

• Big Data Video Channel on SiliconANGLE.tv
• (Forbes) Who Are The Top 20 Influencers in Big Data?
• Wikibon's Complete Archive of Big Data Professional Alerts

Action Item:

Footnotes: