Big Data Tutorial

"Big Data" refers to datasets which cannot be supported by traditional databases because they contain a very large volume data; with a variety of formats and structures; and data that moves at a high velocity. The term "Big Data" originated in 1997; became a hot term; and then cooled in use. Volume, variety and velocity are known as the three Vs of Big Data.

Since the three Vs were identified, more Vs have been brought forward. The 10 Vs of Big Data and Data Science are:

• Volume: super huge amounts of data, measured in the Terabytes and above
• Variety: great differing formats in addition to traditional structured data such as free form text, images and audio recordings.
• Velocity: measure of the speed at which data is coming in to the system. Major websites like Twitter and Netflix have tens of millions of customers who are generating data.
• Volatility: varying useful lifetime value of data. Data may quickly become stale.
• Variability: inconsistencies in data - variations in arrival time or form.
• Veracity: data which is regarded as trustworthy because it has a known and reliable source.
• Validity: accurate and timely data that is fit for its intended use.
• Vulnerability: potential for data breaches which expose private data of persons to wrong doers. With Big Data more personal information can be exposed in a single incident, meaning greater Vulnerability.
• Visualization: showing data through data visualization tools in challenging due data volume, variety and velocity.
• Value: ability to produce helpful results such as better insights and decisions resulting in: improved customer service, better products, greater revenue, reduced cost and/or managed risk.

Big Data is important because it provides input to data science approaches like Machine Learning (ML), and Artificial Intelligence (AI). Use of these approaches have led to innovative breakthroughs. At time same time, the term Big Data is fading as large volumes of data are the norm rather than cutting edge exceptions.

Big Data References and Links

Check out these Big Data links:

• 12 Big Data Definitions : Gil Press, 2014, Forbes
• The 10 Vs of Big Data : George Firican, 2017, TDWI
• The 42 Vs of Big Data and Data Science : Tom Shaefer (Elder Research)