Managing Big Data for Small Business

Send to friend

Every company these days complies a great amount of data, gigabytes of data.

For any business managing data is a challenge, it is also a critical part of managing the day to day running of your business. 

Big data is often thought of as a problem for large companies, however this is a big misconception. Every business needs to manage data effectively and efficiently so that you can get the most out of any analysis.

I have seen first-hand where data has been misused or not correctly managed. The resulting mismanagement of data put very large contracts in jeopardy due to the client not being able to tie up invoices or manage the supply chain through their track and trace system.

Logistics companies possibly hold a great deal more data than most companies. They are managing all kinds of data from packing lists, to custom forms, SKU (single keeping unit) numbers for track and trace systems and customs entries.

Data Quality is your friend or foe

At the logistics company I worked for, everyone groaned when I started talking about data quality. What is data quality? It’s when your data is 100% accurate.  Making sure you have one address for invoicing, one address for each delivery location, one entry for your contract. Basically no duplication of records. If you don’t have this you will find that every part of your management data will give you incorrect results.

Your mobile apps, websites, management reports, client reports will have data missing skewing your results.  It could mean that you have over inflated figures which don’t tie up with your management reports. This then can end up with manual intervention which costs your business more time in man-hours than if you had an accurate process in place.

I have seen businesses of various sizes having the same pains and same issues. These can be easily solved with proper management and processes in place. I would recommend that every small business heavily invests in a process or system to effectively manage data quality. Initiate processes to stop double entries being entered into an operating system. Also CRM (customer related management) systemslinked to a central repository to look at businesses addresses. Real time searches to make sure address information isn’t already in your system.

You may think “yes” this is easy and exists like the logistics company I worked for. It didn’t actually have a robust data quality program. This was due to the various elements of the business divisions having their own operating systems. Each division had a different method for managing data. No CRM systems and account systems were linked. This was through various acquisitions and mergers. I was tasked to build supply chain reports using the track and trace data and to then try and map the other systems to this data. I found very quickly that the reports I was providing didn’t match up to the financial spend.

We found that any operator could create variations of an address which would be linked to the track and trace system. In one instance we found over 180 different versions of one address with the bulk of the invoicing and shipments incorrectly linked to the reporting system. This meant the customer’s reports were only about 20% accurate. I had to put in place a daily report for this one client to reduce the errors. After a while the accuracy increased to 98%. (The 2% was easily explained. However the manpower just to manage this one account was massive. If however this logistics company had put in place an accurate procedure at the start of the contract, then quality would have been high from day one with accurate reporting.

The data quality process for this organisation, once we had an effective process in place, enabled the company to develop new tools based on the data. This was clean and created such reports like Carbon Calculators for all modes of transport and other real time reporting to help different industries manage their supply chains accurately.

With the new GDPR (General Data Protection Regulations) coming out May 2018, this is now the time for any size business to get control of their data and how you handle data. Making sure you know where your data is, how is it managed and processes in place to clean and keep data 100% accurate.

For a small business like a small courier company, then getting someone monthly or quarterly to clean your data, will help your business greatly. In particular from the time you start looking at growing and if subsequently, you are looking to sell then, with a robust data management and controls in place, your business will be far more attractive.

Data will be the biggest asset your business will have in time and you need to treat it with respect.


James Blackman

I started my working life in Road Freight I work through various departments including Marketing/Sales/Supply Chain Analyst/Customer Solutions. Every step through my career in logistics I was involved with cutting edge technology and when I left to start my own Web Design and Marketing company I have used what I have learnt to make my business grow. I am currently the President for Lichfield & Tamworth…

Comments (0)

Add a Comment

This thread has been closed from taking new comments.

Editorial: +44 (0)1892 536363
Publisher: +44 (0)208 440 0372
Subscribe FREE to the weekly E-newsletter