SAP B1 blog

What is Big Data?

Big Data is actually just a collective term. It includes new technologies that enable the collection, use, exploitation, structuring, marketing and, above all, analysis of digital mass data.

What is data mining?

Data mining is the search for "gold in the data mine". The gold that is mined here is knowledge, models, statements, correlations or the verification of hypotheses.

Data mining is often applied to large amounts of data (big data). Often, conventional methods for Analysis of this data not sensible or not possible.

Big Data- The difference to Business Intelligence

Business Intelligence (BI) and Big Data both aim to systematically analyse and evaluate data. With BI, it is determined at the beginning which data is used for which purpose and in which structure. With big data, limitations should be avoided as far as possible. If possible, all contexts that define and accompany an event should be determined and made accessible for evaluation. Logically, the data sets become extremely large with such an approach.

The basis for Big Data is therefore access to data, too much data. Or Big Data. However, it is not only the mass occurrence of data that is characteristic. Certain properties of the data are in demand. In connection with Big Data, one also speaks of the "four Vs". 

1. Volume ? A lot of data

   The main feature that makes data "big" is the sheer volume. The raw material for data mining is not running out, as the total amount of information is growing exponentially every year. Experts estimate that the amount of data stored on servers worldwide will grow to more than 175 zettabytes (a 175 with 21 zeros) by 2025. If this data were stored on conventional DVDs, it could be stacked up to form 23 towers with foundations on the earth, the tops of which would reach as far as the moon
   

2. Variety ? Variety makes the difference 

   More and more data digitised. In addition to the traditional data types (structured data) include, for example, things on an invoice like date, amount and time. These are things that map well into a relational database. 
   This structured data is increasingly supplemented by unstructured data. Such data can be found, for example, in Twitter feeds, audio files, images, web pages, weblogs etc. In other words, everything that can be captured and stored but does not have a meta model that defines it properly.  
   Unstructured data is a fundamental building block within Big Data strategies. The best way to understand unstructured data is to compare it to structured data. You can think of structured data as being well defined in a set of rules. For example, money will always consist of numbers and have at least two decimal places; names are expressed as text, etc. So data follows a certain pattern. 
   With unstructured data, on the other hand, there are no rules. An image, a voice recording, a tweet - they can all be different. However, they express ideas and thoughts based on human understanding. This understanding needs to be understood with technology in order to make unstructured data usable for data mining. 
   

3. Veracity ? Trustworthiness of the data 

   Veracity refers to the trustworthiness of the data. Can the data miner rely on the data being representative? First of all, the origin of the data is certainly important. Internal data can naturally be trusted more than external data. But the care taken in data maintenance also plays a major role in whether data can be trusted. Were there rules, models, processes for data collection that are comprehensible? In short, you should know where the data comes from and how it came about. 
   

4. Velocity - Faster and faster 

   Velocity is the frequency of incoming data that needs to be processed. Depending on the data source, the speed at which data changes can be very challenging. When in doubt, data that is not updated can significantly affect an overall picture. 

What does Big Data Analytics do? 

Big Data Analytics describes various methods and analytical approaches. On the one hand, the analysis of data provides insights and results that can be presented in reports and their visualisation. The data processed in this way should make decisions easier for companies. Big data analytics focuses on searching, discovering and predicting.

Machine learning models are characterised by the fact that software automatically optimises its performance on the basis of a lot of data and thus leads to new insights. Based on this, predictions of events within certain timeframes are then made. processes made. This can concern the early detection of risks and failures or a recommendation on what to offer a certain group of customers.  

The middle class hesitates

A Commerzbank study entitled "Raw material of the 21st century: Big Data, Smart Data ? Lost Data?" revealed that the topic of big data in German SMEs appears to have room for improvement, to say the least. A survey of companies revealed that a hesitant attitude still sets the tone. Reasons for this were possible problems with data protection (30%), the lack of skilled workers (40%) and a lack of motivation on the part of management (31%). And only 8% of the companies surveyed recorded their data systematically at the time of the survey.

Does Big Data benefit SMEs?

The hurdles for SMEs At first glance, large industrial companies seem to have an advantage when it comes to utilising big data. One would think that large corporations would find it much easier to provide the necessary resources to pursue big data strategies. In addition, access to large amounts of data is naturally much easier for large organisations. However, the word "big" in big data is often misinterpreted. This is because even many small data sources (internal and/or external) that can be combined may result in a very good database.

Use cases for Big Data in SMEs
Sales & Marketing - Reduce costs - Increase in turnover - Market analyses - Competitor analyses - Optimisation Point of Sales - Recognise customer motivation - Cross- and up-selling optimisation - Campaign optimisations - Customer segmentations	Product development - Evaluate complex data in real time - New development - Improvements - Trend analyses of new product ideas - Increasing cost efficiency - Remote monitoring parameters
production - Optimisation with machine data - Production planning with real-time data from sales - Early detection of problems - Predictive maintenance of machines – Process optimisation in production	Distribution and logistics - Optimise supply chains - Optimising logistics on the basis of transport telematics - Optimisation of transport costs
Finances & Controlling - Real-time responses - Simulations - Scenarios - Prevention of manipulation - Risk controlling in real time - Identify credit risks

SMEs sometimes have an advantage

However, SMEs have a huge advantage over the big players in the industry: agility. 
Big Data Analytics is essentially about spotting trends and gaining insights to adapt to a changing business environment. Discovering these trends is one thing, but being able to react to them quickly is another discipline. SME speedboats' can do this much better than 'industry tankers' of this kind. data analysis benefit.

Why wander far afield...

Another advantage for SMEs is when they already have a good basic structure of integrated data to refer to. Business software such as ERP systems, which have often been in use for years (sometimes decades), may contain a treasure trove of data that is worth mining. Often, numerous other systems, from web shops to CAD systems docked. The SME also has access to their data, which can lead to new insights when put into context.