Strategic Data Management and Deletion – Post-GDPR, are you going far enough?


Many months on from its introduction in May 2018, GDPR is still dominating conversation.

Organisations may have addressed their immediate compliance priorities however there are still a number of complex data management challenges to be overcome including the crucial question of how an organisation can ensure, and evidence, its ability to comply with enhanced data destruction requirements.

The need to destroy data is not a new concept. In fact, the ability to destroy data at the end of its lifecycle should already have been integral to any strategic data management policy.

With the introduction of GDPR however, specifically the requirements brought about by Article 17 and the associated ‘Right to erasure’ this has introduced, it was unlikely that existing data management policies went far or wide enough to ensure compliance. Under this new article, all individuals (both customers and colleagues) have the right to understand what data is held on them, and if requested, organisations must be able to delete that data. Whilst this is theoretically simple, for most organisations, particularly those larger enterprises that store vast volumes of data, locating and deleting such information can be a significant challenge, particularly in legacy systems pre-dating the data regulations. In short, data deletion requirements have substantially evolved from the comparatively simple batch-deletion of eligible data – typically driven by date/event – to the more customer-centric, reactive deletion strategies which arose from the new rights introduced under GDPR.

There’s no denying that the task of determining which data should be deleted is not a simple one. Organisations are faced with a complex range of what may appear to be conflicting priorities, particularly in Financial Services where a number of different regulations and possible disposal hold notices need to be considered in relation to the same data. On one hand there may be the requirement or permission to retain data relating to – for example – a customer’s credit product, for a number of years after their account has closed; on the other hand a customer may request that this same data is deleted before this period has elapsed. Many organisations’ Data Management and Records Retention Policies have therefore been scrutinised closely and updated to reflect the relative priorities between these requirements.

A recent Gartner article comments that “over the next two years, organisations that don’t revise their data retention policies to reduce the overall data held, and by extension the data that is backed up, will face a huge sanction risk for noncompliance as well as the impacts associated with an eventual data breach.”

Faced with the threat of fines – the greater of up to 4% of their global annual revenue or €20 million – for non-compliance, CIOs will need to ensure that adherence with the essential elements of all regulations is at the forefront of their agenda. Whilst an organisation’s technology should provide the mechanism to enact their updated retention policies, it is sometimes the technology itself that prevents them from doing so.

Here we take a look at the various options available to organisations as they make strategic moves towards data management, data compliance and to mitigate their risk of over retention:

Option 1 – Delete the data!

As the title suggests, option one is the most obvious. Organisations who have the capability can remove all eligible data from legacy systems by undertaking a targeted purge of all eligible data. This can be managed in bulk with deletion strategies based on date or on a case-by-case basis in response to customer requests under regulations such as GDPR. This is the utopian approach which would mitigate over-retention risks and ensure compliance whilst reducing the overall data footprint – bringing about significant system and data efficiencies, including the opportunity to retire costly legacy systems retained purely for the storage of data.

This option is, however, built on the assumption that organisations have a well-documented and manageable data architecture. The stark reality however is that for many large organisations – where increasing amounts of data is stored on a daily basis, often with varying retention periods and with multiple business uses, and with large, complex storage footprints which rely on legacy systems which were not built with GDPR in mind – this is simply not the case. Organisations tend to have primary and backup data stored in multiple locations and across different storage media, both digital and physical. Traditional backup, which has been designed to provide recovery when data has been lost, can be exceptionally challenging to access and navigate. Access outside of this intended use-case can be constrained by the base technology and very often, finding individual records is a painstaking and near-impossible process. In addition, organisations will typically need to delete entire records held on a file rather than being able to identify and select small specific data items within records for deletion.

Option 2 – Pseudonymisation/Anonymisation of Data

A popular ‘workaround’ option used to meet the new GDPR data deletion requirements, and to manage the conflicting priorities between regulatory requirements, is Pseudonymisation or Anonymisation of the data. Using this approach, any personally identifiable information fields within a data set are replaced or masked with artificial identifiers, or pseudonyms. This renders the records unidentifiable or searchable by the individual’s key details, and therefore satisfies the GDPR requirements.

This approach is more suitable for data held in data warehouses, rather than systems used for operational purposes where it may not be feasible to remove key data, and may be preferable for organisations who would like to continue to make use of the data left behind for statistical purposes or for activities such as trend analysis. Where Pseudonymisation is used, a full records purge within the corresponding operational systems may still be required in tandem in order to ensure GDPR requirements are met.

Option 3 – Putting data ‘Out of Reach’

With the constraints of legacy system design on some organisations, another ‘workaround’ option being adopted is to put data ‘out of reach’ of staff. Where it has proved impossible – or too risky for the wider system or data integrity – to delete individual customer records or targeted groups, by breaking the linkages between the records and the front-end systems through which the data is accessed, this option effectively puts the data ‘out of reach’ of staff.

Whilst this is not the ideal solution to comply with the letter of the regulations, it provides a realistic alternative solution in the spirit of GDPR. The data still remains, but can only be accessed programmatically by authorised and suitably skilled staff, so the risk of data leakage is minimal within an organisation also compliant with the relevant Information Security regulations.

For both Options 2 and 3, these tactical workarounds still require investigation, detailed design, development and thorough testing, all of which come at a cost which in these options will not be offset by a corresponding saving through the decommission of legacy systems. And they are not without risk, with unforeseen impacts due to the complexity of the legacy data landscape and business processes not always being fully documented and understood.

Option 4 –Strategic Archiving Solution

For those organisations not fortunate enough to have a full targeted purge capability built-in to all existing systems, perhaps the most forward-thinking option is to consider using a strategic archiving tool. In this option, historic data, and new data reaching maturity can be moved to an Enterprise Archiving Solution where the data remains readily available when required, and can be scheduled for deletion at the appropriate point in line with the organisation’s data obligations, or targeted for deletion at record level upon request.

In addition to ensuring full compliance with GDPR and any other relevant regulations, such a solution can deliver significant cost savings both through the business process efficiencies realised by having access to all historic data in one place, as well as from an IT cost perspective. Following the migration of data required for retention to the Enterprise Archiving solution, costly legacy systems can be decommissioned, freeing up IT budget for more strategic investment.

By utilising a solution like StorARCH, the Digital Storage and Retrieval Solution from Krome Technologies, this can offer an Enterprise Solution to data retention that allows organisations to archive their critical data – of any format – within a secure central repository. Confidential data can then be very simply retrieved, redacted (if required) and downloaded securely as part of an end to end Data Storage & Retrieval process. By utilising StorARCH’s strategic purge capability, data can be automatically purged at the end of its lifecycle or indeed, at the request of a customer in accordance with GDPR.

With the value and importance of compliant data management at the forefront of industry awareness, more and more organisations are looking to truly strategic means of meeting data management challenges head-on. In the past 12 months especially, we at Krome Technologies have seen a real shift in priority from our customer base and there appears to be a growing recognition that tactical, short-term fixes to data compliance will take more time, resource and cost more money in the long-term. By looking at the big picture and helping organisations to really get-to-grips with the true extent of their data footprint, we have been able to offer tailored solutions for large, complex organisations to simplify their overall data management strategies.

StorARCH offers a truly strategic option which can resolve data destruction process challenges at the macro (entire enterprise – in line with data retention policy) and micro level (in response to individual customer requests as now entitled under data regulations such as GDPR). By employing a strategic approach, organisations can ensure that their data management strategies are forward-thinking, progressive and agile enough to support future business requirements as data-consciousness continues to develop and mature.

To find out how StorARCH can help your business achieve regulatory compliance, improve operational efficiency and reduce costs, get in touch with us today or visit for more information.