The structured data records blindspot is biting us all through privacy legislation.

One area which has been liberally ignored by most of the Records profession, is structured data.

Largely, that’s been OK. 

We’ve been exchanging data in static representations of what appeared in a structured data system – and keeping records of that.

It does mean that we’ve endlessly migrated structured data from system to system, without ever destroying it.

And that’s the problem now.

It’s left us all incredibly light on capability in structured data, and now privacy law is exposing it.

I think there are four basic capabilities for most privacy legislation:

  1. Comprehensive and searchable identification of all records associated with a client – structured and unstructured.
  2. The ability to package all client records for export.
  3. The ability to sentence data.
  4. The ability to defensibly destroy.

While the first and second capabilities can be taken care of with good enterprise search, the last two have been mostly ignored for many years outside of the records profession, and by records in structured data systems.

And now we have to do them at scale, under time pressure.

How customer service portals fundamentally change the nature of record keeping.

When we engage with clients now, it’s easy to record the information we provide them.

Because we always exchange a document.

A significant trend though, is the move to portal based engagements.

In many cases, the portal content is static documents held elsewhere – so record keeping is relatively easy.

Dynamic content though, is generally composed in real time from many sources.

With the state of APIs and integration services, these sources could be outside the control of the organisation.

So an accurate recording of the interaction requires one of two things:

  1. Keeping a point in time representation of the information provided every time a view of it is presented to a client.
  2. A way to re-compose the information as it was provided to the client at a point in time, in a verifiable, evidence grade manner.

One of which is easy, storage intensive and doesn’t scale.

And the other, which is amazingly difficult and the only way to scale effectively.

Why we fail at information management when we don’t get policy enforcement right.

Organisational policy is about making clear what the organisation expects.

It’s about giving ordinary workers clear rules about the value of information.

We do this, because information that people will handle often has “no value to me”.

It does have value to the organisation though.

It has value to the officer who spends six months looking for it after a subpoena.

It has value to the big data project that will rely on its quality.

It has value to the compliance officer who will need to show it to an auditor.

But in the moment, the value to the user might be nil. And they won’t care, unless we help them care like the organisation does – with policy.

Which is why we fail if we don’t get policy enforcement right.

Why do we tolerate the fiction of “not enough time to keep records?”

Defining a record is an organisational policy decision.

It’s the organisations way of saying “this information is important enough for us to write a policy to make sure we handle it properly”.

I read an article recently that said workers “don’t have time for drag and drop” – as some kind of excuse for why they weren’t using a records management system.

It’s crap.

We have time for coffee.


Toilet breaks.

Talking to co-workers about the weekend.

Endless meetings that generally achieve nothing.

Spending 40% of our time looking for information.

And we drag and drop 300 times a day.

Somehow when we ask people to drag and drop into a records system, they don’t have time for it.

It’s rubbish, records are part of peoples jobs because the organisation says that the information is important, and needs to be kept. 

If people don’t have time to do their jobs, why aren’t they being performance managed?

If managers can’t get their teams to do their jobs, why aren’t they being performance managed?

Why do we accept the failure to link policy to practice?

The legal trend that every organisation needs to pay attention to, and that a Records Manager can solve for you

Is the change from legally mandated retention of records, to required destruction of records.

Current laws mandate that an organisation must hold specific data for at least a certain period of time.

The trend in privacy legislation all over the world, is to mandate that organisations must destroy it at a certain time, or for certain reasons.

Many organisations that I’m talking to are deciding that data disposal is too complex to implement, and storage is so cheap that there’s just no point.

In many cases, they’re then adopting systems that have no capacity to dispose of information programatically.

It’s a cheap decision now.

It’s going to get very expensive.

For a Records Manager, this problem is routine. It’s just what they do.

Every organisation is going to need the capability to destroy information programatically to exist in the information economy of the future.

Every organisation is going to need a Records Manager.

What use of the word “compliance” can tell you about maturity.

There’s a simple relationship between maturity, and the way an organisation uses “compliance”.

Organisations with low maturity use “compliance” to describe compliance imposed externally.

Organisations with high maturity use it to describe compliance driven internally by organisational policy.

Policy is simply a tool of organisational agreement. Its function is to make clear how the organisation should behave, and to remove individual decision making power about an issue. 

Policy for instance, could mandate that information is only ever to be captured once. Compliance would require information is re-used, rather than recaptured.

If your organisation regularly uses phrases like “it can’t just be about compliance”, they’re both wrong, and missing opportunities to structure and standardise organisational action. Everything can be about compliance, it’s the mark of a mature, deliberative, policy driven organisation, and it can absolutely be a way to win.

Constraints on your Records, Information and Digital Strategy.

The two largest constraints on your strategic options are:

  1. Authority (support of management).
  2. Culture (support of staff).

Authority determines the change you can get funded.

Culture determines to your ability execute change.

The best strategies make full use of the authority and culture of the organisation. 

The worst strategies fail to take them into account – and they fail.