Big head, long tail. Not that I'm stealing Chris Anderson's theory of the web, just re-awakening a description of government on the web that the OeE used in mid-2003 and one that we probably borrowed from Clay Shirky.
When I first worked in e-government, there was only one strategy for getting government online. It was driven by the the "2005, 100% online, citizen focused, joined up government" vision which, arguably, is as much a strategy as you need, but there wasn't anything more explicit than that.
The result was thousands of government websites, very frequently replicating content that was already available elsewhere, still more often providing content that was less than authoritative or was quickly out of date as few had realised how genuinely hard it is to keep a website up to date and relevant.
During 2003 I regularly tracked the number of occurrences of a few government products across the entire .gov.uk domain as a proxy for the degree of duplication. I repeated the test today to give some comparison For instance:
| Product | June 2003 count | July 2007 count | Growth |
| Disability Living Allowance | 9,000 | 122,000 | 13x |
| Tax Credits | 11,400 | 193,000 | 17x |
| Housing Benefit | 49,000 | 1,160,000 | 23x |
| Self Assessment | 22,700 | 401,000 | 18x |
This is a far from perfect proxy but I think it plainly illustrates a staggering rate of duplication, even just within the .gov.uk space. To try and refine the measure a little, I looked at the rate of growth of the same terms within the site of the nominal owner of the product:
| Product | Owner | 2003 count | 2007 count | Growth |
| Disability Living Allowance | DWP | 1,010 | 1,660 | 1.6x |
| Tax Credits | HMRC | 2,730 | 28,800 | 10x |
| Housing Benefit | DWP | 2,180 | 3,900 | 1.8x |
| Self Assessment | HMRC | 2,820 | 41,700 | 15x |
This shows that whilst the products certainly occur more frequently now than in 2003 within the owner sites, the growth is far higher across the wider government domain for 3 of the products (and a little higher for the last, self assessment).
My argument, often expressed in this blog, is that this is wasteful for government and confusing for the customer. In July 2003, there were 2,643 government web sites. I don't have a current total but the last time I looked, about a year ago, the total had increased to over 4,000. I suspect it's not far from that now - given that very few sites appear to have been closed since then.
One of the consequences of confusing customers appears to have been that few used online government services.
At a Public Sector Forums conference in March 2003, I shared this slide with the audience. It showed that individual government sites had low market share and low loyalty when compared with market leading sites (such as the bbc and amazon). When aggregated together, however, the potential market share for government was significantly higher - although loyalty didn't budge much. In short, joined up government online has the potential to reach a big chunk of the market
To illustrate this point further, here are the current market share figures for the top 10 government web sites for last week, sourced from Hitwise via Public Sector Forums (the comparison figures are those from July 29th 2006):
1 Met Office www.metoffice.gov.uk 13.42% (versus 11.48%)
2 Jobcentre Plus www.jobcentreplus.gov.uk 9.89% (versus 10.21%)
3 Directgov www.direct.gov.uk 7.18% (versus 4.51%)
4 HM Revenue & Customs www.hmrc.gov.uk 4.31% (versus 4.49%)
5 Vehicle Licensing Online www.vehiclelicence.gov.uk 2.55% (versus 4.39%)
6 10 Downing St www.number-10.gov.uk 2.28% (not listed)
7 Driving Standards Agency www.dsa.gov.uk 1.77% (versus 2.15%)
8 DVLA www.dvla.gov.uk 1.73% (versus 2.1%)
9 National Center for Biotechnology Information www.ncbi.hlm.nih.gov.uk 1.57% (versus 2.13%)
10 National Archives www.nationalarchives.gov.uk 1.55% (versus 1.84%)
Local government websites, individually, all have market shares of less than this - which makes sense. But, like I said, upfront. Tadpole. Big head, long tail. Or maybe that should be long tale. Here's another view of the tail, from a conference in May 2003, this time looking at pages of content per site (and showing that the bulk of the content is in just a few sites - something like 80% is in less than 5% of sites - although this isn't a perfect measure as some sites hide content behind passwords or database engines).
Back to the top 10 though. Downing Street's site makes a rare entry to the top 10, perhaps driven by the change in PM this month (although they were, I think, at the top during the petition against road pricing excitement). I'm fascinated to see NCBI in there - and even more surprised that it is a consistent top 10 entry. Directgov is, though, the only site to show a significant change in market share - increasing nearly 60%. In October 2005, Directgov's market share was lower still, at 2.8%. It looks like the killer government website might be a site that finds jobs for weather forecasters
Only two government websites make it into the top 100 UK sites according to Alexa. Jobcentre is at 83 and Transport for London is at 92. So a market share of around 10% is good for only 83rd place. Incidentally 10% UK share is 0.01385% of global market share.
There are, then, two strategies for dealing with this.
- The folks at directionlessgov (oh how they must have laughed when they came up with that name. I'd probably have gone for indirectgov - there's plainly "direction" even if there are those who disagree with it) believe that the solution to the vast number of government web sites is to provide great search, by harnessing google. They're widely quoted as having spent �60 and 75 minutes developing the site, not to mention harnessing the $168 billion market cap of google. It's been tuned since then and looks really nice - and it's also added a post code search capability which is very smart.
- My approach and now also that of the Transformational Government programme, is to aggregate content into "one" site and drive traffic to that through changing the balance of government marketing to promote it heavily. Government is complex - it's like the most incomprehensible film you've ever seen with a convoluted plot (The Good Shepherd anyone) with threads that diverge and never get picked up again. Without a guide, you're lost. You sit and scratch your head wondering what it's all about.
Today, both strategies are needed - but should that be the case always? Without a guide, such as the search engine offered by directionlessgov or the aggregation of direct.gov, you'll struggle to find what you need in government. I like what the directionless guys have done and government departments, like many corporations, need people to thumb their nose at it and show it there are other ways - so these comments aren't a dig - but it doesn't solve the real problem which is that tens and probably hundreds of millions of pounds are being spent supporting, maintaining and promoting a vast array of government web sites that rarely see any visitors who almost certainly have very little loyalty - because, for the most part, they don't get the answers that they need.
The problem with search engines is you have to know what you're looking for - what the search engine does is tell you where you can find it. So a search for "tax credits" will absolutely take you to the tax credits site (actually, in google, the number one return for this is directgov) but if you're a new parent and you want to know what government can do for you, what would you search for? Searching for "new parent government uk" takes you to a content aggregation site - parentscentre.gov.uk. Go to direct.gov and you're one click away from a whole section on what government can do for you as a new parent. Likewise, you're a click away from everything you need to know about employment, motoring, travel and whatever else you need.
Direct.gov has, of course, a search engine. But it's deliberately hobbled. It's job is to keep you inside the direct.gov website so it only returns entries from that site. When we built direct.gov there was a huge internal debate about whether it should do that. I was, for a long time, pro a wider search. With ukonline.gov.uk, we'd had a full pan-government search engine (using a product from inktomi). As I've said before we tried to do a deal with google - who, at the time, were extraordinarily difficult to deal with. We ended up talking to Eric Schmidt and Sergei Brin directly before we got any traction. We explored a few options, including using their API and rebadging the results (as directionless does) although the problems with that are several fold:
- We couldn't promote specific campaigns by influencing the search results to ensure that our chosen campaign hit the top spot (although there are ways to do this with sponsored links, it hardly made sense to pay for such a "benefit"). I can't even remember, to be honest, how developed ad words were when we were doing this in late 2003.
- Newly posted content wouldn't be available through search immediately (we'd have to wait for google to index the site and we had no control over how frequent that indexing was)
- Google makes money through advertising and, in theory, had we used them we'd have had their ads alongside our content (directionless strips those which I'm sure is fine for a few dozen searches a week but not so for several hundred thousand or even a few million)
The final deal we worked on was using a "google appliance", effectively google in a box of its own (one that was made up to look like swiss cheese I think), that we'd host alongside direct.gov and therefore have full control over. We would have been the first overseas user and google had no UK presence, so would have helped us maintain it remotely. The price, in the end, made us baulk - Tom has commented that it would have been money well spent, even for a million or so, but at the time there was no way we were going to do it. So we looked at other deals and ending up working with Verity, now owned by Autonomy (I don't know if the new direct.gov still uses Verity). We spent a lot of time with Verity, working towards our original aim of searching all of government. At the same time, the direct.gov (actually, it was called the Online Government Store at that time) folks were building their strategy of having a "closed search engine". Oddly, it was Verity's inability to perform at the scale we needed - it just couldn't return the search results quickly enough when we had a full index of the whole of government - that aligned us with direct.gov's strategy. Verity, despite their best efforts and the involvement of the guy who wrote the engine, couldn't solve the problem. And so we ended up, both by design and by default, with an engine that searched only within the walled garden of direct.gov.
Plainly direct.gov isn't all of government. In it's early days it wasn't even very much of government, but it has grown now to involve 18 different government departments, all of whom assign people to work co-operatively to join up disparate pieces of government policy and process in a way that makes sense for the customer. It is, of course, still a veneer. Government isn't joined up behind direct.gov - and it won't be for a long time but the take up of direct.gov shows that there is demand for it to happen.
As it aggregates more and more content and especially when it bolsters that with transactional services inside the same brand, then the need for a pan-government search engine decreases - because all of the content is housed in one place, in a consistent and sensible way that is easy to navigate. It will never do everything right - the directionless folks note, in parliament no less, at the responses you get if you put the word "goat" into the search engine (google takes you to defra's goat pox page, direct.gov takes you to a page about pregnancy, and not about goat pregnancy - where it advises you not to eat goat's cheese).
Who knows what you're after if you type in "goat" - maybe you're looking for the law on having a goat as a pet, maybe you're after a picture of one, maybe you want to know what grants are available if you want to be a goat farmer; maybe it's a typo and you really meant "boat". Government websites aren't going to learn to read minds. And no single site will, any time soon, hold all of the combined knowledge of government that is distributed over those 4,000 sites and however many hundred million pages.
But, for 80% of the people, it's reasonable to expect that a single well designed and architected site could do 80% of what they need. The rest probably need a good search engine that looks across all of government - and for that, direct.gov could do far worse than link to directionlessgov (after agreeing a deal with google that lifted the cap on search volumes perhaps). I even wrote the pitch for a pan-government search engine, posting it on this blog at the end of 2005 (I wrote it 2 or even 3 years earlier I think) - I ended that post with "Anyway, great search is a necessary part of driving e-government adoption. I just believe that without significant consolidation of websites, it won't deliver what's needed". I still believe that.
In the meantime, direct.gov is doing something right. It's volumes are growing at a hefty clip, reaching just over 4,000,000 a month in January 2007 (and I've heard numbers up to 5 million since then):
In early 2003, I proposed a plan for dealing with the problem as I saw it. I haven't much wavered from this plan as the ultimate way of dealing with e-government strategy, although I've probably written it in a dozen different ways at various times over the last few years.
- Fewer websites not more. Kill 50 websites for every new domain name.
- Less content not more. Delete five (or fifty, or five hundred) pages for every page you write.
- Solve the top 50 questions that citizens ask ... and structure your content around those first. Then do the next 50 and the next. The people who know these questions are the ones that answer the phone in your call centres, the ones that write in to your agency and the ones that visit your offices for help; likewise, they visit accountants, advice bureau, charities and so on.
- Test search engines to see how your site ranks - both from a mind share side and for individual queries.
- Impose rigorous discipline on use of "words" - plain speak.
- Impose even more rigorous discipline on the structure of the content, including metadata so that it's easy to read - by people and by search engines.
It seems to me that executing against this plan will drive out cost from government (freeing up money to invest in new services and capabilities), encourage greater take-up (if every government letter had a single site name printed on it, pretty soon it would stick in everyone's mind), spark the necessary initiatives to join up the back ends (customer pressure would soon call for that to happen) and so drive out back end efficiencies in government. It would even reduce the environmental footprint of e-government. Wins all round.
