In today’s increasingly data-driven world, the principle of 'Garbage In-Garbage Out' has never had more importance as the amount of data being processed continues to increase exponentially. At Impact Data Metrics we have made it our mission to collect, clean and organise data to ensure that we - and our clients - use the highest possible quality datasets for analysis.
Humans are very forgiving when it comes to mistakes in data records - we don’t mind mis-spelling or gaps. But when a machine needs to analyse many millions of records, each has to be accurate and data must be correctly stored and organised. Failure to do this produces errors and bias in results. If uncorrected, the potential damage resulting from bad data can be catastrophic.
These risks can be mitigated by focusing on 3 key data principles - Stewardship, Integrity and Provenance.
In building our comprehensive datasets, we source information from over 9,000 repositories globally. To make this an efficient process, we use Genie™ to provide Digital Asset Management and Active Data Services covering harvesting, ingest, storage and access to data. This is a continuous process of stewardship that currently manages over 3 billion data records.
Genie™ works with Iterative Folding™, our Machine Learning-enhanced mosaic technology that fills gaps and checks and corrects errors in data records. This is an automated process that is the foundation of our ability to provide high-integrity datasets for analytics
Working alongside this process is our TrueProvenance™ technology, which was developed to track data from its source, through use and re-use, and any modifications, augmentations and updates. It is a full life-history of data records. TrueProvenance™ not only provides full audit capability, but aids in ensuring that any analytics processes that are undertaken are reproducible.
© Impact Data Metrics Ltd 2019