If the technology is in the way of records adoption, we need to change the technology.

We have a problem in Records Management.

Technology that we’ve been comfortable using for many years is inadequate. 

It’s been inadequate for 20 years.

That’s obvious to all of us in one way or another.

How many agencies have catch rates at 20% or lower?

It isn’t just a matter of staff adoption – although that’s still an issue.

How can staff keep records about processes that are managed in a structured data system?

When we’re handling it badly, we’re making decisions about what “rises to the level of a record” and deciding that it’s not stuff in that system because that’s not a record keeping system, so it’s very difficult to work with, and generally belongs to IT who don’t understand records.

When we’re doing it well, we’re understanding the underlying technology and what it’s capable of, and writing policy and executing process to ensure that at some point, that data is maintained as useful information and evidence of business activities.

And that’s really the crux of it. 

The old technology isn’t working for everything, so either it has to change, or we have to.

Personally, I don’t think the old tech can change enough to cope with the new world – it’s still a paper paradigm, and we’re already moving beyond electronic paper representations. I also don’t think the new world will deliver the kind of records experience that is “good enough for government work” – because the new world doesn’t understand records, and the underlying principles of records are still the right ones.

We need the new world, and the new world needs us.

We can’t be effective if we have a narrow definition of records.

So we need to change the technology that we’re comfortable with.

We need to understand databases, web servers, middleware and apps – for a start, because people are relying on the information they provide every day of the week, and we can’t abandon them to the idea that someone else MIGHT be able to reconstruct what they saw.

We need to change the technology we’re comfortable with, if we don’t, I’m pretty sure there won’t be a records profession – and then we’ll have to get comfortable with the new technology anyway.

I’d like to think that these are all original ideas, but Barbara Reed, Gillian Oliver, Frank Upward and Joanne Evans were writing about it in RIMPA newsletters at least as early as 2008. Have we learned?

Would a move to mandatory destruction fix a lot of the problems with Records Management (and everything else)?

When you’re working with paper, it’s far easier to bin a record than to pay a team of people to organise and manage it – and that’s why we have records legislation.

Now we’re dealing with a different problem. Rather than being easy to destroy, it’s actually become difficult.

To be fair, what’s actually difficult is creating the conditions so that information is well organised.

When information is well organised, we know we can destroy with confidence.

When it’s poorly organised, the only thing we can do with confidence is retain.

I think that if agency heads had to sign a statement every year to the effect that “we have not retained any information longer than we are required to”, records managers in government would be in a very different situation.

I think we would also find many other initiatives easy – open data, open government, digital, anything involving data science – and everything else that relies on well organised information. 

When we get right down to it, well organised information is a necessary precondition for solving almost every information, data and records management problem in existence.

And it would probably be cheaper overall.

I think it could all start with a change to mandate destruction, because mandatory destruction requires us to organise, and all mandatory retention incentivises is a mess.

How does mandatory metadata affect capture rates in records systems?

This is an outstanding question for me.

Marketers know that there’s a drop off in the sign up rate for every additional field they ask a user to complete.

I’d bet we see the same thing in Records.

The more mandatory metadata that we demand, the less records we capture.

I think this is one of the reasons that there’s often so much enthusiasm for network drives and other systems that don’t demand metadata when people want to store things.

Has anyone ever tested this? Or do you know of good research that has?

The fiction of sentencing on creation, and why the model is broken.

It think “sentencing on creation” creates a problem for records management. It creates a fiction that a sentence is applied that we can actually follow through on. The end result is quite different, and the economics of sentencing don’t change – and the economics of storage have, so sentencing has to as well.

The idea of sentencing on creation is fairly simple.

We apply sentences when a file is created based on the function and activity.

This means sentencing on creation is actually a prediction about what the file content will be.

And we know the prediction isn’t accurate.

So when we actually come to the point where we need to destroy, we can’t rely on the sentence, and we don’t press the button.

So if we end up destroying anything, it’s because we’ve sentenced at the content level.

In a paper world, there was a business case and it made sense to spend time appraising content for destruction or archival.

In the electronic world, that business case just doesn’t exist.

The hundred year cost of storing a document is below the cost of getting an appropriately skilled person to sentence it.

So we don’t do it.

Ultimately, the economics of sentencing and retention have to change if we’re going to keep doing it.

It’s already changing for consumer focused organisations with the consumer data right.

Outside that though, I have had many conversations with regulated organisations that have previously had destruction programs, but are now questioning whether they need to. They know that they can stay in compliance with the law as it stands by retaining indefinitely.

When I discuss this with people in the community, the conversation naturally switches to risks associated with retention. So far though, they’re fairly toothless in Australia. They’re also set to remain fairly toothless outside of a few consumer focused areas.

One of two things needs to happen.

We either need to rethink the model of retention and destruction altogether, or the economics needs to improve.

I think a foundational re-thinking is necessary.

When retention rules were established, the focus was very much on ensuring that evidence was retained for a suitable period. It’s understandable because it’s cheaper and easier to throw paper in a bin than to hire a team of people to catalogue and manage it. The economics of retaining were very much in favour of destruction.

The economics of storage have improved dramatically though, and are improving still. While the costs of sentencing have dropped marginally because there is less manual handling, in the end, there is still a person appraising a document.

Fundamentally, I think that at an organisational level, we all need to recognise that sentencing is an economically heavy activity, and reserve it for classes of record that are actually high risk.

Without a rethink, there are only really two viable options to continue destruction practices:

  1. Apply a “legal fiction” that the content reflects the file sentence – and hit destroy.
  2. Use technology to bring down the cost of sentencing at the content level to a point at which it becomes feasible.

The report missing from every records management system

Is the report about what’s not there.

Why can’t the records system have place holders for the documents that we know will appear as part of routing business processes? Then we could report on when they’ve been filled.

Yes, this won’t work in every case, and email is the hole in the entire information management universe, but when we get right down to it, business processes all exhibit some form of reliability.

There is initiating information, information generated during the process, and then some form of output.

So why doesn’t the records system give us a place to put each of them, and tell us when it’s missing? 

Every freedom of information request has an initiating request document and decision about whether the request is valid.

Every development application has an application document, and a determination.

Every investigation has an initiating event, an investigative log, and a report.

Why doesn’t the records system tell us when these things are missing?

Do you have a plan for the most important part of Records?

The most important part of records, is records culture.

I’ve seen organisations with ten year old, poorly performing records systems keep immaculate records because it was just “how we do things here”. 

When great record keeping is “how we do things here”, staff just do it, and records managers get to spend time on projects that matter to the organisation. 

So what’s your plan for culture?

What every records system specification gets wrong.

Is that it doesn’t focus enough on quantitative analysis of tasks based on their frequency and time taken.

Think of the following two functional requirements:

  1. Must provide for storage of records.
  2. Must provide role based access.

Equal weight.

Here’s the problem.

If it takes 30 seconds to store a record.

And 4 hours to configure a users access permissions.

The cost to the organisation of storing records will still be tens of thousands of times higher over the life of the system.

So we give equal weight to factors of unequal cost and value.

And we’ll give a system that takes 30 seconds, and a system that takes 10 seconds the same score.

And then wonder about user adoption.

Thinking strategically about records and being underestimated.

Are your records good enough?

Good enough for what?

The good news, is that we all get to decide.

If our strategic goal is to tick a compliance box, “good enough” should be relatively easy to answer.

If our strategic goal is to ensure that information is only captured once and then reused, findability matters a lot.

If our strategic goal is to beat litigation, evidentiality is going to be our number one focus.

If our strategy is to use records as the foundation of competitive advantage, “good enough” will be harder still to define. Quality will matter a lot, and it’s likely that structure, placement and a hundred other things will too. 

And if we don’t have a strategy, and we haven’t made sure our organisation knows about it, whether our records are good enough will be determined by every Tom, Dick and Harriet who make unreasonable demands of us. 

And the worst case scenario is probably that they chronically underestimate us, and always get what they want because we didn’t show them a strategy that made us reach for whatever is next.