What is big data?

You may have heard the term ‘big data’ but not be totally sure what it means. This quick guide should help.

So, what exactly is big data?

Big data is essentially large sets of diverse information that continuously grow. The use of the word ‘big’ covers quantity, speed of growth, and variety. Big data is widely used in data mining and machine learning

Where does the data come from?

Big data is collected from a huge range of sources, including social networks, websites, questionnaires, sales and financial transactions, and many others. This data is usually collected with the individual user’s consent – although not always.

Types of Big Data

There are three main categories of data within big data: structured, unstructured, and semi-structured.

– Structured data tends to be information that is stored by an organisation on databases and/or spreadsheets. It is easily accessible and often mostly numeric.

– Unstructured data tends to be more qualitative and less organised. Examples include free-text answers, messages, social media posts and IoT sensor data.

– Semi-structured data has some of the characteristics of each.

If you feel that a data collection company could help your business, a data specialist such as shepper.com can offer guidance, as well as a range of data services.

Where is it stored?

Mostly due to the sheer volume, big data is usually stored electronically in data warehouses and/or data lakes. These tend to be cloud-based. Data warehouses are also able to analyse the data and run complex queries on large data sets. Many major companies offer software-as-a-service (SaaS) packages which include storage and management of the associated data.

How is big data used?

The various pieces of data are analysed by specialists who try to identify relationships and correlations. Part of their role is also to get all data into consistent and comparable formats.

Findings from the analysis of big data can be used by a variety of business departments such as HR, sales, and marketing. They are particularly useful for new product development, personalisation of offers and understanding consumer behaviour.