“You will never work in a place like this again.” (The Office/BBC)
In the local government office, pinned on the cafeteria wall, was a poster. “Do you struggle to focus in meetings?” it asked. “You might find that something like knitting could help you.” I suppose that for those who just can’t concentrate on the task in front of them, the inclusive thing for an organization to do is to pay them to do something else.
As a data scientist in the property sector, my first mistake was to ignore these blandishments and insist on trying to achieve something. In 2022 I joined the housing department of one of the London borough councils. Our ostensible purposes were to procure and manage temporary accommodation, to use that accommodation to support homeless applicants, and to find and prosecute rogue landlords. We had also recently absorbed another social housing body and their staff, and were trying to pull these people and teams together into some kind of integrated service.
The challenges we faced, and the tools we had to face them with, were similar to those of hundreds of other local authorities across the country. In the last financial year £2.8bn was spent across England on temporary accommodation, and our own trends in cost were no more alarming than the national ones.
This week, in keeping with Andy Burnham’s drive for devolution, the Government announced that mayoral authorities will gain powers to call in planning applications and manage public assets. To discharge these new powers, they will lean heavily on councils as their source of information and skills. It is worth taking stock of who we are ultimately devolving power to, and whether they are capable of exercising it.
In the council office we began each office day with a team ritual: swap mice and keyboards between each of our desks to fill in the gaps and assemble a working computer. IT struggled to replace broken equipment, so we counted it a good day if four out of five computers were usable. Wiser heads knew that if you pretended to have fibromyalgia, a computer would be provided for you in advance. Once you had a working machine, you might be forcibly logged out until you had worked through an HR instructional video, in which cartoon characters and multiple choice questions explained that you shouldn’t leave your laptop out in public. Then at last you could face the horrors of the day.
It started as soon as we picked up a call. The system on which we logged our call records didn’t stop you from saying that a tenant had 38 children, or was 120 years old, or had left temporary accommodation before they’d entered it. So we created more new errors each day in our records than could ever be corrected. Those who find modern life excessively preoccupied with identity would have had a field day with our contact databases, in which a host of columns (Gender, gender, GENDER) had been laid up to store our many genders.
Because we did not have control over this database — and could only amend a single case at a time — we invested ever-mounting effort into training dozens of staff to fix errors manually. Every quarter, they would trawl through our web portal to find and fix, one by one, the small fraction of errors that stopped us making our statutory data submissions to central government. The rest were left to multiply. My suggestions that we fix this broken system and check the data before it was put in were not considered sufficiently accessible or focused on the resident.
People who deal with databases like to use the phrase “garbage in, garbage out”, meaning that if you use them to store any old rubbish, you might not enjoy having to reach back in to retrieve it later. In my experience this understates the problem. As time passes things change inside a bin, and not for the better.
These errors had consequences. Although the amount of housing benefit paid out by the council to temporary accommodation tenants was compounding year on year, we had not taken the trouble to find out how many tenants we were responsible for. The two figures available came from two different databases held by two different subcontractors. Both were irreconcilable, and both wrong. The figure we used for reporting in each case depended largely on whether we wanted to convey a reassuring vibe or a worrying one.
Likewise, because the council had lost the history of our social housing applicants during a migration from one software tool to another, it was impossible to say how long people had been waiting for social housing. No regular check was made on the employment status of our temporary accommodation tenants and, being unable to access data from the Department for Work and Pensions, we could not say what proportion of our tenants were in work. As with the number of tenants, this plurality of truths was often convenient; a grim figure might help us seek more grant money, and a cheerful one demonstrate progress.
Such shoddy record-keeping might have remained a mere embarrassment. But over time it led us to become increasingly unmoored from the reality of our tenants. The result was taxpayers’ money misspent or simply thrown away. For example, we had no process for closing the accounts we set up to bill tenants. One consequence was that the “temporary” storage facility we offered tenants to help them move house was used by some for years, or decades, as a permanent near-free service.
We did not know whether half of our tenants had any local connection or real need to be placed in our borough. Rather than enquire into their circumstances, we kept them in expensive locations. Often these were hotels – though I cannot say how often, as these placements were sometimes paid for by card and recorded in a separate, unknown place. And our system for recording rent payments equated four weeks to a month, with staff turnover too high for there to be any institutional memory of why. I never could find out whether we were therefore charging far too much or not enough — every month, to every tenant.
Our management of public money was no better in big things than in small. This was exemplified by the way the organization handled procurement. Out of curiosity, I requested a copy of our contract with the company that provided our tracking software for homelessness support cases. I discovered that it was about to come up for renewal. No one in the organization was aware of this renewal, whether it was desirable, or on what terms to make it. In the rare cases when an independent decision about data was made, it was made by managers who did not know the difference between a database and a website, were unaware of what data systems already existed, and had no understanding of the need for common identifiers to link different systems together.
Because no single individual knew how the whole system fit together, vagueness reigned. No matter how many people you asked, the truth about who ran a system or how it worked was typically considered ineffable. All I know is that the handful of staff with actual experience of data were never party to big decisions about it. As one analyst put it: “This is one of the countless times my working life has been made worse by decisions about databases which I had no involvement in.”
The role of the data professional, then, was manually to extract wrong information from a broken system they were banned from fixing, omit anything either too flattering or not flattering enough, and arrange for it to appear ephemerally to an innumerate manager in a pointless meeting.
What lay behind this culture of obsessive inattention to detail? Partly, behaviors that undermine public administration everywhere. Rather than pressing forward, bureaucracy tends to grope around laterally. For instance, local government is obsessed with imitation. In our case, it was the function of the “policy officer” to find out and copy anything another public body was doing, especially if it had an intriguing title with the word “strategy” in it. This approach led us to continually update a gray, churning world of unread documents. There would always be a policy document that needed a refresh, or to be turned successively into half a dozen briefings at different levels of abstraction. “This strategy is really more of an action plan.” “This action plan is really more of a strategy.” The continual push by management to make a document the same as that produced by every other public body, to the point of lifting whole sections, made perfect sense in a realm where copypasta had become the only form of credibility.
Local government is also an enthusiastic promoter of “everythingism”, a tendency to require one policy in one area to solve all problems everywhere. It was considered “siloed” to focus discussion on our basic responsibilities, but “open-minded” to link homelessness to climate change or call for total equality of outcomes across society. All meetings took on a standard form: describe every activity currently being conducted as “siloed”, call for a “holistic approach”, and make plans for a further meeting in which these lines could be repeated.
But questions of data are too specific, too technical, to allow for this. The result was a counter-norm we might call “nothingism”. That is, while every area of public policy must be discussed with reference to every other area at the same time, nothing concrete could actually be established, because obtaining data about anything was made impossible. That is not to say people were incurious; in fact they were often very interested in what others were doing, in case it might be useful in their personal quests for power. I recall being upbraided by a junior colleague for being slow to update my office calendar when my wife went into labor. But when it came to sharing information about their own projects, a cloud of unknowing would descend.
Still, people are easier to change than ideas. It might have been hoped that new hires would gradually replace the existing blob with a more advanced generation of staff. But these were not the people sought. In assessing candidates for senior data roles, the criteria were whether they could recall niceties of legislation without prompting, not whether they had any data skills, were competent to procure IT tools, or knew anything about programming. Steady wage compression did not help. Pay settlements in which lower-banded staff get a percentage pay rise, and higher earners a smaller lump sum, have left data staff salaries hopelessly uncompetitive. Whether or not a market exists for policy officers, there certainly is one for data engineers and database admins, and it pays more than any local council.
This creates an incentive to avoid any training in data skills that might lead to council staff being poached. Instead, training is only offered on the aforementioned bespoke, barely-functional tools that have no value to a prospective employer. The result is that programmers are replaced by staff who can perhaps query a database, and these in turn by staff who can hit “refresh” on an old report but lack the skills to create anything new at all.
I eventually accepted that no effort or stratagem was going to simultaneously change hiring, procurement, management, or a general culture of disinterest in whether our programs were reaching the right people or having any measurable effect. Nor did a major systems failure seem likely to cause an awakening; the investigation we were under for widespread maladministration had led only to our spasmodically procuring another parallel IT system.
After three years of fruitless attempts to map (never mind fix) these multifarious problems, I decided to lay the job aside. I went back to doing what I had done before: building data projects in a young team which survives on grant money. It’s true that it’s easier to build a new system than fix a broken one.
Until now, this is how the Mayoral authorities have operated. New, and with a continuous need to justify their existence, they haven’t calcified as councils have into an impermeable bureaucracy.
This should be a reason to welcome one of the Burnham reforms: the granting of powers to regional authorities to overrule local planning decisions. Ironically, the Burnham devolution agenda may work best where it makes public services less local — by allowing mayors to snatch them from unreformable councils!
The rub is that they will depend on local information to do it. The most generous incentives and strongest mandates are likely to founder if they rely on organizations with profound cultural commitments to inefficiency. The only chance of success is to establish entirely new systems and allow the old to wither on the vine. And the old system doesn’t deserve to survive.
Tenants in bedsits died while we debated whether their social housing priority should be low or medium. The deliberate policy of fragmenting our information across half a dozen broken systems made their personal experience a mystery.
Ultimately, there was something evil in this banality. The culture we worked in amplified our worst instincts and thwarted our best. It took momentary mistakes and turned them into permanent rot, and took a genuine desire for joined-up government and sublimated it into pointless meetings.
Did we do any good? Perhaps, behind the fog, we were making a genuine difference to the lives of vulnerable people. Or perhaps, as the critics say, we were simply allocating workless households to prime London real estate. With no knowledge of where our applicants ended up, we are unable either to chastise failure or to praise success. The truth is impossible to trace.