Friday, December 28, 2007

Bo And Luke Make Glass, Wine

image The Dukes of Hazzard was a TV show in the late 70s and early 80s. The plot, at least the way I remember it, always involved Bo and Luke, two brothers, getting tangled up unexpectedly in some nefarious scheme cooked up by the local Mayor, Boss Hogg. A sub-plot always involved Daisy Duke, their sister, wearing shorts and running around in a jeep, but hey, I was a teenager back then. At some point, the boys would end up crossing the state line and so be free of arrest by Boss Hogg. That "crossing the state line" has featured in many hundreds of American TV shows and movies.

It always seemed to me that governments are not unlike the Dukes of Hazzard. Not so much the smuggling of moonshine and driving fast cars, although I guess that happened in some places too, but in the idea that there's a "state line" and when you cross it, your problem becomes someone else's or, at the very least, there's a whole new set of law makers involved. Cross the post code/zip code barrier and whether it's healthcare, drug availability, housing taxes, bin collection, dog poo removal or whatever, it's different. In most countries that I've visited, the consequences of this are near identical business processes supported by [deliberately] entirely incompatible IT systems across many dozens or hundreds of operations. Needless to say, almost every process lacks the scale to operate effectively and efficiently. The drivers are maintaining local control (or the illusion of it) and ensuring local people take local jobs for local citizens.

I'm always interested in people or businesses that break out of the mould of "we're different, we need our own process/system/operation/call centre/sales and marketing operation" etc. Working in banking a decade ago, every country-centred business had its own operation and own IT - its own FX books, own securities settlement system, own cash reconcilement process and so on. Those were gradually simplified, rationalised and operated at scale. Sub-prime loans notwithstanding, banks operate pretty efficiently now, at least in their transactional operations.

Two examples of breaking the mould that I've come across whilst out and about (@large?) in the last few months:clip_image001

1) Glassmakers in Murano, under pressure from competing (and they'll say inferior) products from elsewhere in the world (notably, but not limited to, China), have started to merge to gain scale. Visiting one factory on a recent trip to Venice, I saw that they made only one type of product - very modern. I asked about their other products. The manager told me that they had recently merged with 14 other glass makers, with each one deciding to specialise in just one product area. They had figured out who of their masters was best at each product and then given them the job of producing the very best of that product at a volume that the market can support. With the process from novice to master taking 15 years or more, and young people increasing leaving Venice to work on the mainland, there's also a shortage of talent - and so no longer the ability to support every glass maker producing every type of product. Together, they put their money to work to build a single showroom that displays all of their products. Each one bears a seal of quality and the signature of the master who produced the piece. Prices are clearly displayed - well, clearly until you hesitate whereupon the calculator comes out and an "off-season" discount is proposed.

image 2) Winemakers in Australia, under pressure from the effects of climate change, their appreciating currency, the massive competition in the wine industry at the price mid-point (despite Australia beating out France in volumes, the bulk of the sales continue to be at the low end of the market - and the French are now starting to re-work their marketing and pricing and will, if they aren't already, gain ground), and keen to show their products in the best light, have started to form alliances. One such alliance, Artisans of Barossa, brings together a dozen individual producers, all of whom make unique and special wines, and who now market their wines as an ensemble. Tastings are arranged head to head - so you can try out, for instance, an out and out Shiraz against a more varied Grenache, Shiraz, Mourvedre combination. None of these wineries particularly needed to come together - their wines are good enough to sell by themselves (and often have the awards to show for it), yet they recognised that together they are stronger: they have scale, can reach wider markets, can pitch each other's products, can learn from one another, can reduce their costs of marketing, shipping and representation and so on. Taste their wines if you get a chance - and if you don't, contact me and I'll tell you where you can buy them.

I know that governments at various levels have tried this - whether it is local, regional and sometimes even national departments - but there doesn't seem to be, from where I've stood and looked, the same willingness and engagement. Sure, the pressures are different, but the thinking should be the same.

If a local government is recognised as having the fastest, most efficient housing benefit process, why wouldn't councils in the area (hey, even the country) say, "I'm not that good at HB, but I am good at business rates - why don't I give them my HB and I'll take their BR". I understand that charters would have to change, I understand that system modifications would have to be made - but surely those are not beyond the wit of man if it simplifies and rationalises the processes. Not every process is unique and special - or, in fact, not every process is necessarily unique and special - I see that they often end up that way.

We could take this to a national level - and I've rambled about this before. The department of give, and the department of take, for instance. Payment scale and receipts scale. Could it work?

Sunday, December 09, 2007

How many armies does an e-government need?

clip_image001Whilst @large over the last few weeks, four apparently unrelated events fused together in my mind to create an idea for governments that might make for both some fun and some real business benefit. These are the four events:

1. Since I first heard about it, probably when I was 10 or 11, I've wanted to visit the Terracotta Army near Mount Lishan in China. The nearest I've got so far is the exhibition at the British Museum. I'll take that for now - seeing the original Tutankhamun show in London was the pre-cursor to seeing it all for real many years later after all. There are many astonishing things about this army - the scale of imagination to originally envision it, the incredible craftsmanship to produce such individualised warriors (coupled with an enormous army of people to make them), the bureaucracy and managerial process to create it (I won't dwell on the facy that most involved were probably killed right after completion), the damage done to it not long after the first emperor died and, now, the reconstruction effort that means we can at least see some of the pieces pretty much as they were in 210BC - this last thought is only truly appreciated when you see the stills of how things were when they were found: millions of fragments piled one on top of the other with little to differentiate them. It felt like there were 100s of people in the Reading Room at the British Museum on the day I visited, snaking in long lines from exhibit to exhibit but I suspect they restrict each visiting slot to 50 or 100 people at at time. It's truly an impressive draw, although one that leaves you longing to see the entire spectacle.

2. A few weeks ago I was amongst the first to know about a newsworthy event - and I found out through public sources rather than through some devious internal channel. It turned out to be a big story but I suspect few realised it at first. I happened to think of going to wikipedia to see what it said about the event. It was silent, entirely unaware, it seemed, that anything had happened. I took the liberty of adding my footprint to the armies of those who have gone before, and edited the appropriate page with the updated information. I sat back, pleased that I had added a [very] little knowledge to humankind. Within 15 minutes, seemingly dozens of others had updated the site, refining the information I posted, adding citation and links to other sources. The space that this news topic occupied could initially have been seen as very niche, yet a veritable army of people were apparently looking for something to happen so that they too could be editors of their own newsfeed.

3. I went to a meeting with some people who know do, loosely at least, some of what I used to do in the Cabinet Office. Of course, they've more than moved on from what I was up to - it's around 2 years since I left. But a lot of the topics we discussed were ones that I'd spent time on before; ones that I'd commissioned work on, even paid money to allow government to action them in perpuity. I took along a document that my team had produced, with a vendor, in late 2002 when we were looking at rebuilding ukonline (now directgov) for the 3rd or even 4th time since its launch in early 2001. The document was bristling with great ideas on how to engage the citizen more, how to expose more of government to the outside world, how to structure websites and transactions so that they'd have the most impact and what areas to concentrate on first. It was a great piece of work and whilst we'd acted on some of it, I was sure that more than 50% had been left undone for time, money or capability constraints. In truth, armies of consultants, IT vendors, outsourcers and business process experts compile hundreds or even thousands of such reports every year for government as it merrily spends around £3 billion/year on consultants.

4. Lastly, I was looking for some figures to tell me how much use was being made of Freedom of Information requests. When I first thought about this law, in 2000, I was expecting it to be the offline equivalent of the 1901 Census website - something that would knock government out as it responded to potentially millions of both frivolous and fact-seeking requests filed by armies of citizens and, especially, journalists. As far as I can tell, it's done nothing of the sort. But the more I hear about FoI the more concerned I am about whether we've taken the right approach in the UK.

So taking those four un-related things into account, I wondered:

  • What if government took facebook into the inside? What if we ditched every intranet there ever was in every government department and allowed everyone to create, instead, a facebook page for themselves? The same tools and applications would be available; groups joined would be centered on areas of expertise & experience (desired or actual) and room to play would be allowed to - no point in making it all business, there needs to be some kind of trade. Straight away, links would form between people doing similar jobs in different parts of the government (or different parts of the same department but spread around the same country); experience would be shared; job-postings would be easy to find and could be matched by a talent inventory that could draw on all 4-5 million public sector employees (that number could be anywhere from 250,000 to 7 million depending on how you cut things). Now I'm no great fan of facebook - truth be told, I don't really get it - but I get its potential, in a slightly different context, to replace the intranet - to be a place where people look up contact information, find people that might know something that they need to know, exchange holiday photos, date, arrange to meet or whatever they need to know.
  • What if government took a licence for wikipedia and built an internal version? What if that site became the place where all reports from every consultancy that's ever worked for government was published? Where people edited topics that they were interested in and added statistics, links and sources that were verified by the armies of others that were also interested in those topics? What if this became the hub of knowledge were people found out how to do their job, what they could do to develop in their job, where they would find information from others doing the same job, where they could see what consultancies and others had recommended could be done to a given process, function or organisation in another, related part of government. Or even a completely unrelated part of government. Many of those reports, the many hundreds every week, month or year, end up gathering dust in a cupboard somewhere. The very best are 50% implemented with the remaining actions getting swamped by the pressure of time or money, or the clean sweep of a new broom coming in with different ideas. That leaves perhaps a billion and a half worth of ideas left unimplemented every year. That's a lot of intellectual property left on the shelf. And let's not wonder aloud, at least not here, how much of those reports are repeats of what has already been bought and paid for by a government department somewhere else.
  • Next, what if we took every FoI request - and its response - and published it online with a simple search application, driven by google or windows live or any other engine- so that before you asked your question you could see what else had been asked that was similar; you'd then either just use that information and not bother to ask your own question or you'd refine yours to get a better take. Smart journalists would use the search tool to bring together previously unrelated questions and draw even more conspiratorial conclusions. Smarter ones would phrase their next question to take advantage of the freely obtained knowledge that they already have to find something new. Government would respond, one would hope, by getting smarter about its operations and processes and would use this leverage to drive greater change and efficiency.
  • And lastly, maybe all of this would be turned inside out and put online, not just FoI requests, but reports and consultancy work that government had paid for, so as to act as the single greatest source of pressure for change and, dare I say that ugly word, transformation (the single best example of which continues to be Optimus Prime in Michael Bay's recent Transformers film). The deluge of information would be enormous. The fragments of data would require an entire army to stitch it together into meaningful conclusions. But, let's be honest, government itself is never going to have a big enough internal army to do this stitching but, the outside world, those who want to be part of an open-source government, now maybe they'd have the willing, the time, the intellect and the energy to sort, distill and publish the very best pieces - and government, of course, would pay for such pieces once and once only. Sadly, the name YouGov is already taken by a very clever chap called Nadhim Zahawi, but maybe he'd be open to offers. Failing that, we could always go back to, the vision of access to government coined in 2000 following the [necessary] demise of

This way, the vast body of knowledge that government accumulates year in, year out would be available not only to all of government but to all those with interest in what it says about where their taxpayer pounds, dollars or even, one day, renminbi. After all, it was that first Emperor of China, who unified the country, standardised currency & axle lengths and introduced many other reforms (and yes, I know he killed the 700,000 people who worked on his tomb, but bear with me - the metaphor nearly works).

Over the next 7 years, some 40% of government's workers will retire. They'll take an awful lot of knowledge with them. Not all of it will be useful, but figuring out which is and isn't is a job for a distributed network of staff and citizens who can argue amongst each other, for a while at least, about relative value - promoting those items that their successors need to hold on to and relegating those that they don't. And, in 100 years or 500 years, what better place for those who come after us to look for how things were done back in the early days of the 21st century.

The nice thing about these projects is that they could be started individually and cheaply. There's no need for a huge infrastructure, no need for a complicated requirements gathering process, no need for expensive outsource deals. There just needs to be a bit of willing for a few senior folks in a few key departments who want to give it a try - who want to be bold (but not too bold) and take a step in a new direction. Along the way there would be pitfalls, there would be screw-ups but there would be successes too. And those successes would quickly build as more players came to be involved. Just starting one of these projects - say, facebook as government's intranet - might go further to creating some joined up government than anything that has gone before.

Wednesday, November 28, 2007

Tick box to skip a year

aminny Much to my surprise my entry to the London Marathon in 2008 was accepted in the ballot.  The last time I got a place like that was for Paris in 1999.  Every other marathon I've always run under a charity's golden bond scheme.

The acceptance form has a useful box which allows me to defer my entry until 2009.  With my knee still not working properly after I tore the meniscus back in March, despite an apparently successful operation, this looks to be my only option.

I'm going to give it a couple of weeks before I send in my deferral but I'm not confident of getting from zero to marathon shape on a dodgy knee in the time available. In fact, I'm not sure I could pull off 200 yards right now.

Monday, November 26, 2007

A better version of the "data spiral" slide

Finally found the original of the slide I was looking for from the earlier post, The Data Spiral.  This should be more readable.


It paired up with this slide


Sunday, November 25, 2007

The Data Spiral

Here's an extract from my Government Enterprise Architecture paper from September 2003:

The principal now is that data can be exposed to many viewers – internal staff, third parties, intermediaries and the citizen/business themselves. The number of data sources has been dramatically reduced, perhaps not to one but to a few at least. This has been achieved principally through abstracting the original back end systems using clever technology known as web services and through creating a set of consistent and reusable components.

clip_image002[5]The journey to such an enterprise architecture is lengthy – even achieving such a vision in a single department is a huge challenge. It may be appropriate to think of progress being made along 4 axes, not necessarily with equivalent speed. The axes are business process, business application, business data and technology infrastructure - note that the focus is on business involvement and leadership, especially around such important areas as data.

The model might look like the figure at right. Progress is made by moving out along any of the axes, with the time to make progress and the potential for cost saves increasing the further out you move. Although, progress need not be equivalent against each axis at the same time, there will be points when the next level of change can only be achieved when enough, dramatic progress has been made across each axis.

That slide doesn't look too clear in the blog and I can't find the original to paste in.  I'll keep looking for it and then expand on this post.

But the recent HMRC (and everyone else's) data problems reminded me of some of this - I was trying to create a future model for government technology where the citizen would be put back in control of their data, there would be common (not consistent) processes in government and an integrated suite of technology built with re-useable components.

Governments were, and are for the most part, in the middle of the middle: individual processes custom built each time, data held within individual applications, multiple over-lapping business applications and multiple technology infrastructures even within single departments.  No wonder data exchange is hard.

Saturday, November 24, 2007

The essence of commitment

I had a strange conversation with a guy this week.  I needed him to sign up to do something.  He didn't want to do it.    His way of telling me this was to say that he "didn't want to commit in case he had to decommit later" - of course this was by email not an actual, whatchamacallit, a conversation.  I'm wondering if that should be de-commit.  Or, actually, I'm wondering if there's even such a word.  Surely "commit" means just that - to commit to do something means you'll do it, come what may. Much more than a maybe, somewhat more than a promise, as good as a guarantee from a reputable manufacturer?   We're talking about saying you're going to do something and then doing it.  His name, in case you're wondering, wasn't John Kerry.  Although you can imagine how he'd have explained it to me had he committed: "I committed to do it before I de-committed from doing it."

That narrowly beat a conversation earlier in the week that included two words I never thought I'd hear next to each other: "hardcore strategy."  I have no idea what that means. But I think I need to add it to my CV.  Along with the phrase "always committed unless I need to de-commit, in which case I promise to inform you in writing no later than 30 days after I've already de-committed."

Thursday, November 22, 2007

25 million green bottles

iStock_000003735726XSmall There are, as you'd expect, 1001 stories about the loss of 25 million records relating to children and their parents.  Child benefit is one of the most "taken up" government benefits - something like 98% of parents (umm, sorry, children) receive it (versus perhaps 80% for child tax credit). So there's certainly a large number of people affected - the figures of 7.5 million households and 25 million people total look about right.  I've seen this called "DataGate" by the Independent.  Perhaps "Shutting the DataGate after the horse has bolted" may be better.  The story definitely isn't over and I'm sure, barring any other major news developments, it will hold space in the first 2 or 3 pages of newspapers for several weeks and several more instances will doubtless come to light.

If you have a child under 16, your personal detail (name, address, bank account, date of birth and national insurance number).  It's unclear whether if you used to receive child benefit (i.e. your child is now older than 16), your data was still available on the system, but I suspect not.  Likewise, if you are one of those who are generally off-system (certain members of the military, the police and so on), I suspect that data was held elsewhere - so those who talk about the risk of protected identities being compromised are probably wrong.  It is, sadly, one of the hallmarks of IT the world over that data is held locally in each application for each purpose - so this kind of data exists in dozens of applications across every unit of government, whether central or local, state or national, metropolitan or federal.  When we built the Government Gateway, we looked hard at the data we needed - for instance, to post the PIN, we needed an address; but, once posted, we didn't need it anymore.  So we issued a query to the relevant government back end system, got the address, and then dispensed with it as soon as the envelopes were printed.  But that was relatively easy to do in designing a new system from scratch.  Most systems have been around a lot longer.

Let me state two things up front:

1)  Loss of sensitive data is not just a UK government problem or even just a UK problem.  It's prevalent all around the world, in corporates and govenrments, and made ever easier by the increasingly wide access to email and the Internet - and, of course, by the ever increasing number of systems that store all the data that they ever need right in their main database.  It's almost like we should be surprised if our data isn't out there in the wild world.  Never mind worries about putting some personal information on Facebook, your data is already on several other sites, for anyone malicious or maligned to access.  There's a reason that whenever you see people in a film going into a secure nuclear area, there are two of them and they each have a key that has to be turned simultaneously.  Putting control in the hands of one person can be a recipe for disaster. This latest issue comes on top of:

    • An event just a couple of months ago when a disc being sent to Standard Life and containing details of 15,000 people was lost (sadly also by HMRC)
    • 94 million Visa and Mastercard accounts exposed at TJ Maxx
    • Bank of America's loss of backup tapes containing credit card information for 1.2 million Americans
    • The exposure of the records of 800,000 people at UCLA
    • Reed Elsevier's loss of personal information on 300,000 Americans
    • Transaction data for 180,000 customers of Ralph Lauren
    • The use of unsecure email to send out classified nuclear secrets (that's a link to the story by the way, not to the actual secrets)
    • Choicepoints loss of 163,000 individuals records (and the accompanying ID fraud)
    • Hackers in Ohio Universities systems took 137,000 records of students and alumni
    • The loss of doctor's personal information on an NHS website
    • The loss of 26 million records for US veterans
    • and, golly, I've just found this extraordinarily comprehensive list of data breaches.

2) This isn't a problem about why weren't the CDs encrypted or why wasn't the data sent by some other, presumably safer means, it's about several lengthy failings in process: who can access the data, how easy is it to get a full database dump, what controls are there on writing data to CD, who needs to approve what and so on.  In the technical world that most of us operate in we're used to a window popping up and saying "hey, stupid, are you sure you want to delete that entire list of folders and files?".  There is no "are you sure you want to send this data by post dummy?" dialogue box, but there would have been checks and balances before it got to that stage.

It must have been a long chain of events to get to this point.  A full download of every data item in any of the government's big systems isn't the kind of thing that can be just asked for - I'd go as far as to say that it's a one time request requiring special work (although it's possible in this case that the extract had already been prepared for some other reason in the past - and, if that was the case, perhaps many of the usual controls would have been bypassed in this case.  Imagine the conversation "you need an extract? Well, normally that would take us 3 months but I just happen to have one over here, only one previous careful owner, that we took in April 2007"). 

I'd bet that there isn't a requirement in the specification of any government system anywhere in the world to be able to "hit f12 to dump database to two CDs", password protected or not.  So my assumption would be a change request is raised, the IT supplier (probably EDS as the Child Benefit process and accompanying systems used to belong to DWP but were transferred 4 or 5 years ago to HMRC but I don't think they were absorbed by the CapGemini contract) does a quick check to see how long it will take, the change request gets approved (not as quick to get done as it is to write - perhaps a month or more), the data gets offloaded at the next convenient point in processing and then copied to two CDs by someone technical.    Lots of people get involved in this process.  There would even have been a discussion about the cost of removing some fields, hashing out others, creating dummy data and so on.  In the end, it sounds like we've got a very big spreadsheet secured by a password when you try to open it.  I'm not even sure that old versions of Excel can handle that many rows so maybe it was just a word file.  That's a lot of pages.

My guess it that encryption wasn't asked for because the person doing the asking wouldn't have known much about that and the people receiving the data would have known even less, and the technical folks would have wondered about it but would have been busy and so moved on. PKI isn't part of the default desktop installation of any where in government outside spooksville.  I could get into this a lot more but it's a long time since I worked at the Inland Revenue and even then I wasn't that close to the systems involved here - and I'd be speculating.  Doubtless someone is already working on a report and it will come out under FoI or through the persuasive nature of various journalists and, I'm sure, a series of Internet message boards.

As far as I understand, no one ever actually asked for a "full copy of the entire child benefit database".  The NAO asked for a sample of de-sensitized data.  Typically that's a few tens of records with personal identification information removed - certainly the NI record hashed and probably the bank details removed.  When I did a stint in audit back in my banking days, a typical sample was 30 records - statistically, that's enough to give you a sense of whether everything is in order when you're doing a substantive test.  I'm not sure what NAO were trying to prove - maybe that only appropriate data was stored (perhaps that only parents with children under 16 were in the system?) or perhaps that the fields contained the right data and in the right format (post codes matched what they were supposed to) or maybe they were testing that the population claiming matched the expected population claiming.

Putting aside then the issues of should the data even have been floating around 0r what process breakdowns were there, here's a take on the technical aspects of how data should be shipped around:

Most people - as did one commenter on an earlier post - will be asking "why on earth is data being shipped on CD in this day and age?"  A perfectly reasonable question. And one that when you look at the other ways that were probably immediately available, you might briefly think "oh, I see why they'd do it that way" ... right before you clap your hand to your forehead.  Don't think that government (generally, not just the UK) are endowed with the latest hi-tech gear available to one and all.

Two CDs is a fair chunk of data.  At least 1.2GB based on standard format of 600MB a disc.  Not much compared with the capacity of the average ipod (even my iphone has 8gb, I think the entry level classic is now 80gb) or even the average memory stick (2gb is a common size for Vista ReadyBoost).  But a lot of data to ship around nonetheless.

Let's take email as one option - most people would consider that first:

  1. Email systems in government generally have very small mailbox sizes. A few tens of megabytes is very common, even as much (as little?) as 200mb would be uncommon.  This is not like google where you get a couple of gigabytes or more on signup.   Trying to send 600mb would bust both sender and receiver.
  2. Bandwidth between departments is relatively small.  More accurately, there's lots of bandwidth along the backbone  that links departments, but individual links to that backbone are typically small - 1.5MB/s, sometimes less (and are set as a function of the size of the department - I'd expect NAO to be one of the smallest (and I'm actually pretty sure, but not certain, that they're not on the GSI), HMRC to be one of the largest).  Network performance in offices is load dependent and likely to be slow making uploading an attachment of 600MB to the server interminable.
  3. Many government staff don't have access to email at all (if they are routinely processing citizen tax transactions, it's felt there's no need).  Likewise, even fewer have access to the Internet.
  4. Firewalls on the email systems limit attachments to 2mb, sometimes 4mb, rarely much more than that (there are exceptions but they are rare)

But had these all been overcome, the file would have moved between HMRC and NAO within the secure network of government departments known as the GSI.  Risk of interception would have been low (the GSI is regularly penetration tested and is built to a high standard).  But, realistically, this wasn't an option for anyone in HMRC. Government email systems are just not built for files of this size - and I believe that even those that the rest of us use day to day would fall over after trying to digest a file of 1.2gb.  My entire PST file in outlook is only about one gigabyte now (and it has 2 years of email in it, the rest is in archives). With all these issues - and the continuing sense that e-mail is somehow unsafe (like all things on the Internet) compared with "sending 2 CDs by post (!) - I would not be at all surprised to hear that CDs by post is the default choice for exchanging even relatively small amounts of data between departments, agencies and 3rd parties (such as pension companies and banks).

Sometime in 2002 the team I ran in the Cabinet Office built, on behalf of the Criminal Justice folks, a secure email system.  It was the brainchild of the same guy that thought up the Gateway as a pan-government authentication system and, I think, ukonline (which was known originally as  It was designed to allow lawyers working on criminal cases to exchange, securely, documents between their offices and the courts (and each other).  Remote users could use a web-based email front end or their own outlook client and everything inbetween would have been encrypted and secured.  At the time we deployed it, the common way to send such data around was to fax it (you remember the way it used to be done - you'd phone them up, say "stand by the fax machine", then they'd put the phone down and go to their fax, nothing would happen because it was out of paper, or it was already receiving someone else's 100 page fax, all on that slightly fuzzy thermal-style paper).  It was a comedy and needed to be sorted, hence the requirement for the secure mail.  This solution was made available to the whole of government, but take up was low.  I'm not sure that this would have been any better - it would have had the same limitations of bandwidth, firewalls, and so on.

In our own team, and before the secure mail system, we also used various commercial products to exchange secure data (the systems we built and ran were at least restricted and were sometimes higher).  They were based on hosted servers.  But the same issues of bandwidth, firewalls and so on would have applied.  On top of that, both parties have to be connected to the  secure system - so there has to be a set up process: passwords, keyfobs and so on need to be exchanged in advance and kept current. All of those things complicate the issue enormously - especially when such exchanges are not routine and day to day.   What usually happens is that they fall into disuse, the processes breakdown and then rather than take the time to set them up again, people look for a quicker way - popping 2 CDs into an envelope and putting them in the mail for instance.

So, no, email isn't a viable alternative for large volumes of data.  In fact, uploading and downloading to websites via secure spaces, even when encrypted and super-protected, probably isn't a viable way of shifting data around outside of your own secure network within the building, except when you're talking about project-type information and using sharepoint or similar tools - and when you're moving data that you wouldn't mind someone else finding by accident if you haven't set up your server security quite right.

Lots of companies offer solutions to these - the usual products chasing a problem to solve.  There will be lines of them queuing up to offer their services to governments (globally) and their IT suppliers over the next few weeks.  They will offer super-duper-extra-double encryption, they'll say that they can identify rogue data being sent by email and divert it, they can check staff activities on the Internet and make sure they're not doing things, they can spot people trying to download data off a system and copy it to their iPod and so on.  Of course, they spot the problems they're design to spot; not the ones that happen off the beaten track or where the procedures are deliberately over-ridden.

But, on the face of it, had this data been copied to an iPod and hand-carried to where it was going and copied on to another iPod, we might never have known about this.  So iPods to come equipped with a government-approved fingerprint reader as the next step?  Or maybe personal memory sticks with dual control - sender and receiver fingerprint readers.

This is an undeniably serious problem.  There may have been many serious breaches as noted above, but few have stretched as far as the child benefit data.  The solution isn't, however, simple.  And it isn't about secure ways of exchanging data - at least not initially.  There's nothing to say that had this data not arrived at the NAO securely, it wouldn't have been left on an unsecure laptop and then been stolen from the back of a car for instance.


  1. All of the processes around access to patient, customer, taxpayer, citizen etc data in every department, agency, non-departmental public body and local authority are going to go through a rapid review.  New standards will be enforced: senior management sign-off, dual control (keys round the neck and everything), IT supplier held accountable for where data is put and so on. This will take time and still things will be missed and it will happen again - let's not hope that it's on this scale, but it will happen again.
    • Lock down data exchange now.  People come to the data, not the data to the people. Until better processes are in place, this should stop the problem from getting worse.
  2. All staff should be taught the "green cross code" of using computers. The very basics need to be re-taught.  For that matter, the code should be taught at schools, colleges and libraries.
  3. The spooks should lead a review of deploying encryption technology to departments holding individual data so that all correspondence is encrypted automatically in transit using appropriate levels of protection for the job.  This will be expensive.  The alternative though is to make encryption optional - but because you can choose, sometimes people will choose not to (because it's too slow or something) and the problem will recur.
  4. Systems being architected now and those to be architected in the future will look at what data they really need to hold and for how long and will, wherever possible, make transient use of data held elsewhere.  The mother of all ID databases would be a good place to start.

All of this will take time.  In the interim, managers in the line of fire are going to have to use common sense and check and recheck when they're asked to provide information to anyone.  Social engineering is alive and well after all.

Tuesday, November 20, 2007

When things work ... and when they don't

Yesterday, I had a "blue screen of death" on my recently upgraded to Vista laptop.  I guess the laptop did what it was supposed to do: the screen went blue, then filled with a load of hex numbers as it dumped cores or something and then it rebooted.  So far, so part of history.  The surprise came when, after the reboot, a dialogue box popped open and asked me if I wanted Windows to fix the problem.  Sure, why not?  How nice to be asked.

Windows said that there was a problem with my NVIDIA graphics driver (I had no idea I even had one) and provided a helpful link to their site.  At the site, NVIDIA asked which graphics card I had (still no idea) but offered an option for it to find out.  An applet was duly downloaded that found out what it was but sadly (I'm sure it was sadly) informed me that it was a custom driver for a Sony laptop and I'd have to go to that site.

Another link followed, a driver successfully found (actually lots of drivers related to Vista and my laptop).  None of them would download however - bizarre error messages in the middle of the download saying "not a valid filetype" kept appearing.  I left some feedback on the site to say there was a problem.

This morning, the download worked fine and my NVIDIA graphics driver has been successfully updated.  Let's hope that's the end of that blue screen.  Now, if I could just get the laptop to turn off the bluetooth light that has stayed permanently on ever since I upgraded to Vista, even though I have deinstalled the bluetooth drivers.

Wednesday, November 07, 2007 - Everything you wanted to know but were afraid to ask


DSC00001If you've arrived at this page and you're not a regular reader, it's probably because you've typed "" into google or some other search engine (I'd give good odds it was google though) and found that this site has 2 of the top 10 results. But it isn't what you want.

Here are some alternative links to try that I think will get you closer to what you need:

  1. Dubai's "Official Portal of Government", in arabic
  2. The same site, in English,
  3. A page on visa requirements (English),
  4. Flight information for Dubai's international airport (English),
  5. Sheikh Mohammed's own website (English),
  6. The UAE's e-government website (English),
  7. The UAE's e-government website (Arabic), - if you compare these 2 sites, you'll see that the entire arabic site is right justified. Nice touch, obvious I suppose when you think about it.

I hope that helps you with whatever you were looking for when you inadvertently landed at this site.

Online Services - Voter Registration

Today's error trying to get my voter registration sorted on my local council's website:

Microsoft VBScript runtime error '800a01f4'

Variable is undefined: 'sql'

/CXfinalconfirm.asp, line 31

Funny. I don't even know what that means of course.  You'd have thought that would have come up in testing.  There isn't, though, a "report this bug" to me button.  Even better, given they have my voter number, how about they auto-mailed me to say that they'd seen the problem and would fix it and let me know when to come back and try again? Bound to be my fault of course - silly me using a Mac and Firefox.

Paula Gets Gold

image What an awesome run by Paula Radcliffe on Sunday.  After standing at the finish line of the London Marathon earlier this year and watching five hours of what seemed endless pain and suffering, including some (literally) staggeringly emotional scenes as runners collapsed yards from the finish, I'd pretty much vowed never to watch another big race again.  But with Sunday's New York Marathon, the benefit of distance and the aid of television to keep an emotional check on me, I relented. I'm sort of glad that I did.

There were real tears in my eyes as I watched Paula near the finish.  Never will I know how it feels to lead an entire race and finish in 2:23, let alone 3:23 I imagine, but I'd like to think I knew all about the grit she showed as she dropped the hammer and went for the finish on tired legs.  Watching Paula surge past Gete Wami brought feelings of enormous emotion, not to say relief. Bring it on Beijing, we have a champion ready and waiting.

The oddest thing was that it turns out that even when you watch a marathon on the small screen, the last 800 yards is definitely much closer to a mile and a half. It's just amazing how long it seems to take to get through that last part of the race - running or watching- and the tension in New York was incredible.

Tuesday, November 06, 2007

Offline Demonstrations Lack Impact?

On Sunday I was cycling briskly along the river past Westminster when I was stopped in my tracks by a cavalcade of whistles, drums and people in fancy dress. Sadly there were no horses that I could see. This had two immediate effects - a two minute spectacle that was really quite entertaining and then, far, far better, the opportunity to cycle along the river from Westminster pretty much to Tower Bridge without so much as a car on my side of the road.

The cavalcade was, I gather, in protest against NHS cuts. The organisers say some 7,000 people turned out in support (um, protest I mean). Apparently this was a lower than expected turnout because all the senior people in the unions had expected an election to be called on November 1st and so had put plans for the protest on hold. I'm wondering how many other things have been ahem, blamed on the lack of an election.

I was on my way to the Tate Modern to see "Shibboleth", Doris Salcedo's new installation. Or, perhaps more accurately, de-installation. Somehow, Doris or someone Doris knows with a succession of power drills of varying sizes has drilled a long, parallel sided crack from one end of the Turbine Hall to the other. Not only that, but the sides of the crack give the impression that the rock was torn apart - with overlapping edges and bumps sticking out and recessing on each side. It really is a stunning "thing" to look at - and there were thousands there on Saturday, looking, touching, poking and prodding. You can imagine the Tate Board of Directors reacting to her pitch: "So you want to cut a hole in our floor?";"No, not a hole exactly - a crack, from one end to the other"; "A crack? Let me just think about that for a second". How on earth she got away with that I have no idea - but the result is quite brilliant. You'll spend more time trying to figure out how it was done than the average person spends solving the 43 billion billion combinations of a Rubik's Cube. A Shibboleth, by the by, is something - usually liguisitic I think - that identifies those from one region/country/place from another (check the Book of Judges for the detail on how this can get abused). These days it is just as much used to distinguish between those from the new wave of doing something versus the old wave.

image As I cycled down the empty Embankment, en route to the Tate, I wondered about the efficacy of such demonstrations. Sure, people come from all over and parade through the streets of London, making a lot of noise and having a lot of fun (several looked to be having too much fun and having stopped off at the Walkabout for more than a few drinks were lying, in full regalia, in the street looking decidedly the worse for wear), waving placards and so on. London stops for them, as it should - nothing here about people not having a right to protest (I walk past Brian Haw's modern day Greenham Common most days of the week and absolutely believe he should be allowed to do what he does, although I wasn't so keen on all the extra tents recently).

But what does it accomplish? A brief moment in the news - so little news coverage that I was forced to swipe the image above from the daily star website (and I am grateful for that - the image links to the original page). An ephemeral memory for most who saw it. A hangover for some the next day? Shared war stories for those who march to talk to their colleagues back at at work? I still remember the day, aged 9, that I marched for "Save the Seal." Fat lot of good it appears to have done.

There are almost no mainstream news stories about the NHS rally that I can find (using simple search terms "save NHS", "NHS London 3rd November 2007" and so on. Lots of local news stories from the various unions that sent people though.

And then we have our online equivalent - the petitions page:

  • 1.7 million people signed the virtual petition against road pricing
  • 223,000 have signed one asking that the Red Arrows be able to fly at the Olympics (who knew they wouldn't?)
  • 69,000 want student loan repayments to be deducted monthly not annually (that sounds sensible - I might sign up myself)
  • Since Gordon Brown mentioned the petition site in his conference speech, 22,000 have signed up requesting that Jeremy Clarkson be made PM.

These online "rallies" (hardly cavalcades I suppose) seem to attract greater support - the barriers to presence are far lower of course - and they get returning media coverage. On all but the busiest news day there is almost always a story about one petition or another and, as the Daily Mail proved with their campaign against Road Pricing, real "foot" traffic can be attracted very quickly.

Have we arrived at a point where an online rally is a better, more effective and more likely to succeed vehicle than a colourful, noisy demonstration in the centre of London?

When the banks started to change the charges made on student accounts, it was an online group on Facebook that caused them to change their policy again. If Road Pricing doesn't make it, could it have been the 1.7 million e-signatures that made the case stick?

Or will we still hold rallies in real life, as opposed to the second, virtual life, because they're just more fun even if they accomplish less? Is the nature of protest now distinguishable by the shibboleth of whether you protest online or offline?

Thursday, November 01, 2007

Comparing apples with round, green, tasty, slightly acid things

I read today that the price of white truffles, a delicacy from northern italy, has leapt to £7 a gram. One commentator noted it was like "grating a gold bar over your papardelle."  That's quite a lovely image - although I'm guessing a standard grater wouldn't quite cut it so to speak.  It's also wrong.  There are 31.1 grammes in a troy ounce (a troy ounce is a little more than a standard ounce.   My screen shows gold trading at $791 per ounce and the pound is presently $2.07. That makes an ounce of gold £382; and an ounce of white truffle £217.7.  So eating white truffles is like "grating 56/100ths of a gold bar over your papardelle."  Not quite such a lovely image.  But it's funny how often I read an article that makes some egregious comparison so as, apparently, to aid our understanding.  9 times out of 10, it doesn't work for me.

Last week I read that someone had grown a huge pumpkin and that it weighed as much as fourteen and a half Kate Mosses (I hope that's the plural).  I don't know how much one Kate Moss weighs and a comparison between a pumpkin and 14-odd KMs really didn't help me figure out how big it was.  The same newspaper carried a story about the Rubik's Cube world championship and noted that the cube has 43,252,003,274,489,856,000 combinations.  No clever comparison there.  Surely 43 billion billion would have been easier to write? And just as comprehensible.

Why am I wittering on like this and when am I going to get back to talking about e-government? It's coming ...

I was just sent a link to an article about the Government Gateway - one of the best projects I've ever worked on (and one that is still going - which I can't say for everything I've ever worked on).

It contains a splurge of statistics along with some just plain weird comparisons.  Did you know that the weight of the servers in the Gateway is the same as 42 John Prescotts?  No, I digress.  Here are the real ones:

  • 11 million citizens and companies are now e-enabled. In context, that’s equivalent to 120 new Wembley Stadiums, filled to capacity. Or, in other words, an online UK community of registered users, seven times greater than Barclays Bank
  • On its peak day 150,000 people logged on to the Government Gateway to file their tax return, that’s the equivalent to the population of a town like Oxford, or the number of people at the Glastonbury festival in 2007.

120 Wembley stadiums? I don't know what that means.  Seven times Barclays Bank?  Surely that's not important - the Gateway is a monopoly. Barclays has competition. How about "25% of the number of users Facebook has"?  Every day Oxford as a whole logs on?  Reminds me of the commentator during the Rugby World Cup who said, I think about Samoa or Tonga, that "these guys are fielding a team from a country that is so small that it would be the same as Chesterfield entering a team in the competition".  I'm no nearer knowing how many people that means, but at least I know the population of Oxford now, ready for when I'm asked one day (in 2001 it was 134,248)

And one that I quite like

  • 20 million online forms electronically submitted since its inception in 2001. This is an impressive statistic when you consider that if each form had been submitted in paper, via first class post, this would have cost over £6 million in stamps alone.

Now that's got some legs.  £6.4 milli0n in stamps - the cost of a 5,000 square foot apartment by the river.  That's pretty good - and let's not even mention the 60 million bits of paper saved (1 envelope and 2 pages of form) - one for every person in the UK.  Think of the trees. And the reduction in shoe rubber for the postmen.

Ages and ages ago - 2002 maybe - I did a triple header presentation - me, Paul Kelsall (then IT director at the Royal Mail) and Marc Andreessen (needs no intro).  I joked with Paul that he had a real problem looming because government was responsible for about £1 billion of the Mail's revenue through sending and receiving forms - and once e-government took off, they'd be in serious trouble).  Plainly we're a loooooonnnnggggg way from that, but it's nice to know we've made a dent the size of a riverside apartment.

All comparisons aside, I'm as pleased as punch (a hulking Ali-esque one) that the Gateway is alive and kicking 7 years (as long as some wars) after we started work on it (yes, September 2000) and wish only that the numbers were quite a lot bigger. Perhaps we could set some new targets, by 2010:

  • 25 million users - 6 times the population of Los Angeles
  • £16 million in stamps, the same as the number of copies sold of Elton John's greatest hits (and Hotel California for that matter - see an earlier post on hotel - you can check in, but you can never leave. use the search box to find it)
  • 60 million forms (one for every man, woman and child in the UK) or indeed, the number of Vista copies sold in the first 5 weeks (or 30 times the number of Leopards sold by Apple, no not real leopards, in the first 4 days).

There's so much opportunity for comparison.

Tuesday, October 30, 2007

Fry talks Fones

Thanks to TUAW, I found Stephen Fry's blog today.  Despite claiming to be somewhere in deepest (flattest?) Norfolk, far from the realms of Edge let alone 3G and Wifi, he's posted a great (and long) Blessay (his word) on smartphones. He's also writing a weekly technology column for the Guardian.

Who'd have thought I had something in common with Stephen Fry, apart from a rapier wit and a penchant for writing ridiculously long (and infrequent) blog posts I mean. To give that context - and to stop your mind leaping to far wilder (and imaginary) similarities or penchants - he says, amongst many things, for instance

"I've never met a smartphone that I haven't bought"

On Palm's Foleo "If it’s got a chip in it, and a keyboard, and WiFi and a screen and I haven’t sent off for one, then by God you’d better believe it’s in trouble"

On Palm's threaded text "Add to this the inestimable pleasure and benefit of SMS threading (it simply STAGGERS me that no one until now, with the arrival of the iPhone, aside from Palm, offers this, to my mind, essential feature, something that’s been available on the Treo since the get-go)"

On WinMob devices the syncing software, an almost useless PC app called ActiveSync, allows no control over the syncing process and therefore gets any power user into trouble with duplicates, the bane of our lives.

All the big guns want an iPhone killer. Even I, mad for all things Apple as I am, want an iPhone killer. I want smart digital devices to be as good as mankind’s ingenuity can make them. I want us eternally to strive to improve and surprise. Bring on the iPhone killers. Bring them on.

It's a long post from Fry but worth the end to end read.

Friday, October 26, 2007

Apple & Microsoft, Microsoft & Apple

image Over 10 years ago, Microsoft plunked down $150 million to buy a stake in Apple.  The share price back then (January 1997) was about $3.50. Today Apple is trading at $185. That's around $7.9 billion in today's money. Over the same period Microsoft stock is up about 50% (including it's near 10% rise today).  I imagine Microsoft ditched that stock long ago - there was even talk at one time that they might have shorted Apple on the day that they bought it to hedge any risk. Folks have even been saying recently that if, instead of buying a 5gb iPod on the day it came out, you'd bought the same value of Apple stock, you'd have over $10,000 today (looking at the graph, even if you'd bought in 2001 when the iPod came out, you'd still be sitting on 18 times your money at least!).  Apple's certainly on a tear - it's now worth more than IBM and more than Intel.

Today, the day that Leopard comes out, I finally upgraded my workaday Windows laptop to Vista - I'd been running Vista under bootcamp on a MacBook for a while but it wasn't getting much exercise.  I started the upgrade with trepidation - the rumours about it being a difficult install are rife.  They weren't kidding.  The first install trashed my PC and left me no option but to reformat and install from scratch (the approach everyone who'd already done it had recommended in fact).  This worked smoothly.  Restores from backup went fine.  Only one driver crapped out on me.  All of my software works.  And my PC appears rock solid stable and fast - which is better than it was the day before. 

I'll be installing Leopard on the Macs in the house over the weekend I imagine.

Moment of Conversion

Ok, so yes, I'm an "early adopter."  I'm definitely on the left hand side of the chasm when it comes to all things technology.  It's always been fun to try something new and shiny whether it was my first encounter with the Apple II (1978), a CD player (1985 - when Dire Straits released Brothers in Arms), the Apple Mac (1989, especially with Wingz, the first graphical spreadsheet I'd seen, possibly the first spreadsheet as I had no idea what Visi-Calc was at the time), going online (1994, with a 28.8 modem, via Compuserve), my first digital camera (1995,  Casio QV-10), an MP3 player (1998, Rio PMP300),  DVD (on the US release of Saving Private Ryan in late 1999),  WAP (1999, Nokia 7210), the iPhone (2007) or any of the numerous other gadgets I've bought, used for a while and, often inevitably, rejected.  Some of them do, of course, stay the course - there's a "moment of conversion" when you get what the gadget does and that's it for keeps - the iPod is a brilliant recent example (owner since 2001).   In the end, some get replaced by the next version of the same thing - the Sony Ericsson P800 was replaced by the P900 and then the P910 for instance; the iPod 5gb, by the 10gb, the 20gb, the mini, the Nano v1, the Nano v2 and so on.

image When I first saw Sonos advertised, the moment of conversion was instant.  Its premise is simple - any music you have, anywhere in the house.  You can have the same track in every room, all perfectly synchronised, with the volume low (avoiding the usual problem of a one-room stereo where the volume has to be up high to be heard throughout the house, but is deafening in the same room as the stereo); or you can have different tracks in every room, just as you want.  You can use your own speakers or ones that you buy with Sonos.  Music streams from your PC, your Mac or, as in my case, from a network attached storage device.  So I became, as it were, an early adopter of Sonos - it's been installed around 3 years now.  As an aside, talking of "install", the set-up process is absolutely the simplest I've ever seen for any potentially complicated gadget.

Now as a proven early adopter, there are some things I just don't get. and their ilk for instance - I have pages here and there but I just don't get it, too crazy, too random, too hard to find anyone. Perhaps I'm 15 (some would say 20) years too old for a personal page that does anything more than say "hello world". I also don't get - sharp intake of breath from many - blu-ray or HD-DVD.  Sure it's fine for Planet Earth and the Blue Planet or anything by David Attenborough but, so far, it doesn't tempt me - it will do one day, probably when multi-disc-playing devices are available (funny that HD-DVD plays old DVDs by default but Sony, as is their way, went with proprietary non-backwards compatible). I also don't get the Wii.  Hysterical fun the first time, with a bunch of friends - but then so is Dance Dance Revolution in the arcade or Guitar Hero.  But longevity? Not for me. Not even worth a second play yet; unlike Sony's PS3 which isn't actually worth a first play.  So it's not every gadget that gets me hooked.  One final thing that I hadn't got at all was "all you can eat" music subscriptions.  Until now.

With the latest release of Sonos software (like the iPhone, the nice people there issue pretty regular updates that, so far, always have new features - sometimes they let me try them out before the mainstream release which is also nice), comes bundled a 31 day trial of Napster's all you can eat service.  What this means is that any music you have, anywhere in the house moves to any music you want, anywhere in the house.  I won't pretend that it has everything - most online music services are pretty short on classical music as one example.  But it appears to have an awful lot that I want to listen to.  It also has features like "artist mix" where you select an artist you like and it auto-plays, all from Sonos, music that you might like based on that choice.  It's not perfect - I found it playing a little Mary Poppins alongside Beethoven the other day - but it's always worth being introduced to new music (although perhaps not MP again for a while).

For £9.95 a month once the trial expires, I can keep hold of this service and perhaps never buy another CD again.  Or, perhaps, never have to worry about having a backup of the music that I am listening to for when my nice, shiny, 3rd iteration of NAS goes bang.    There is, though, a drawback.  Napster doesn't let you copy the music to an iPod, although you can copy it to just about any other MP3 player that exists. The other drawback is that I don't actually own anything at the end.  No different from leasing a car.  Just trade it in for a newer model when you're bored with it.

Tuesday, October 23, 2007

Lijit Widgets

I've had a search widget on the right of this site for a month or so now.   It's provided by Lijit who I read about on Brad Feld's website who I came across after I met Charlie Feld, his uncle, and a senior guy at EDS. And so the world turns.

I love this widget.  Every week they send me a mail with the stats of who is searching for what, what they clicked on, how they got to the site (if they used another search engine - and some 300 people a week get to this site from google).  There are loads of numbers in the mail - of course they could all be made up, but they look good and they're better than any other numbers I have, which is always important.

Here's an extract from the last 25 searches (which would have been Sunday):

And the top eight searches that brought people to my site (last week)

  1. dlva (15 times)
  2. (5 times)
  3. (4 times)
  4. francois xavier diet coke man (2 times)
  5. mindshare public sector (2 times)
  6. carly goose (2 times)
  7. Generally Regarded as Safe dyes- list (2 times)
  8. importance of egovernment (2 times)

In the train of the goat, I often wonder (a) what people are really searching for and (b) how they got to me.   And then I find out why

  • (1) DLVA is a typo in a post from January 2003.  It's the number one instance of that particular typo it seems (nice to be first in google for something). Of course, google does suggest that you might really want "DVLA", but either 15 people are as confused as I was that time back in 2003 or the same person is very confused
  • (2) And (3) I'm also the top answer for - this search has been used a few dozen times in the last month (I can see that now I have lijit - in fact, I could see it before in the site stats, but they weren't mailed to me in this simple format).  I really should write some more about Dubai's e-government in case it's a real person looking.
  • (4) A topic on which I'd hardly expect to be deemed an expert by google - I have the 6th ranked site even though my post is nothing to do with what's being looked for
  • (5) 4th ranked topic and probably a bit relevant to what was being asked
  • (6) Carly goose, again 4th ranked, although I was talking about Carly Fiorina and wild goose chases.   No better than the top ranked article which is about Mother Goose and, separately (I hope), Carly Simon
  • (7) A post titled "Generally Regarded As Safe" with no mention of "dyes" that I can find
  • (8) 1st ranked.  Who'd have thunk? If you want to know how important egovernment is, come to this site.  Delighted I am. Sure it won't last also.

Great to have good data.  The challenge of course, is to do something about it so that people have a shot at finding what they might need, when the search is relevant.  But also shows how bizarre the search algorithms can be at determining "importance".

A guy standing next to me by the office lifts today started to tell me about how complex "lift algorithms" are - and how specialised and highly paid a career it is.  I've often pondered whether there are such things as lift algorithms given how long I spend waiting for lifts; apparently there are.  There may be a lucrative career awaiting for failed search engine algorithm workers.  Or vice versa.  Or even for me.

Titling the Untitled- Redux

A while ago I wrote about my experience booking a flight with British Airways, specifically the bit where it says "who are you?" and, evidently far more importantly, "what is your title?"  I've pasted in a couple of shots of the drop down menu presented.  I don't think anyone believed me when I posted about it last time.  So here they are, a little hard to read  ... a section of A to B and G to H including "Her Highness", "Air Marshall" (of which, one imagines, there are only 1 or 2 - how nice to build a website that is so personalised), "Bishop", "Graaf" and, of course "High Chief"

image image

Monday, October 22, 2007

Lost Lessons Learned

Seen on a whiteboard in a vendor's conference room, under the bold headline of "Lessons Learned",  two lonely bullets:
  • Lessons learned in the past had not been applied
  • Expectations were set too high

'Twas ever thus

Sunday, October 07, 2007

The 3G Myth

Whilst writing my triptych "the only iPhone review you'll ever need" last week I spent a fair while looking for data that showed actual usage of 3G services in the UK.  I tried various searches in Google and Windows Live (a search engine that has come on leaps and bounds recently and that I really quite like), but most of them returned links to the contents pages of reports published by various doubtless august research bodies, all of whom wanted thousands of dollars to get at the actual data.  All I was looking for were two numbers - how many people have 3G phones and what percentage of revenue from those phones is made up of 3G usage.  I didn't get either of those numbers or anything close to them.  Sometimes the Internet is absolutely the last place you're ever going to find a piece of data that you need unless you work out exactly the string of words that someone else has used or you know a specific site where you can find the data based on recommendation, stumbling across it or paid subscription.

Friday's Evening Standard may have come partially to my rescue(and who said the dead tree industry didn't have a place any longer?).  There's a little graph in the top left corner of a page titled "Mobile is the new battleground in internet gold rush". It shows the following figures for percentage of actual and forecast ownership of 3G handsets by region (the figures are approximate - the scale isn't very detailed):

  Now 2010
USA 30% (or maybe 12%) 70%
Western Europe 30% (or maybe 12%) 70%
China <5% 25%

I was a little confused I have to say.  The graph clearly shows 2007 figures at around 30% but in the text it says "even today, little more than 12% of mobiles in Western Europe are 3G".  So it's one or the other.  Or maybe some other equally made up number.

The focus of the article is really about advertising on mobile phones and, sadly, I don't mean stickers attached to the 'phones promoting products but the misplaced idea that I want (I'm sure someone will use the word "need") banner ads or, god forbid, proximity ads popping up on my phone.

So after spending all those billions on licences, perhaps the mobile phone operators are a little sore that Apple has forced at least one of them back to 2.5G (or even 2.75G if the right upgrades are in place) technology, that is, EDGE.

Wikepedia, ever proving its value as a place to at least get real numbers (although source data/links are not always available) says

"By June 2007 the 200 millionth 3G subscriber had been connected. Out of 3 billion mobile phone subscriptions worldwide this is only 6.7%. In the countries where 3G was launched first - Japan and South Korea over half of all subscribers use 3G. In Europe the leading country is Italy with a third of its subscribers migrated to 3G. Other leading countries by 3G migration include UK, Austria and Singapore at the 20% migration level. A confusing statistic is counting CDMA 2000 1x RTT customers as if they were 3G customers. If using this oft-disputed definition, then the total 3G subscriber base would be 475 million at June 2007 and 15.8% of all subscribers worldwide"

By that reference, I guess UK penetration is likely nearer 12% than 30%. And this version of the iPhone isn't going to help increase that.  Keep focused on SMS you operator folks, at a few hundred quid a megabyte, that's where you're going to make the cash for a while longer.

I was just sorting out some files today and found some old notes that I'd written on a tablet PC - probably the Compaq one that had a detachable keyboard - and printed out for "safekeeping" (I had terrible problems with that PC - I'm sure they culminated in me throwing it across the room one day as it lost some files for the nth time).  They were for a presentation in January 2003.  I included this graph based on the scribbles in my notes (these were made-up numbers - I imagine I couldn't find any research back then either) - but I thought that wifi would be bigger than 3G by now and that MMS would be smaller than both (figures are for users not, say, volume of text sent):


And I suppose this graph shows why I'm not in the research industry

Saturday, October 06, 2007

Do Testers Really Thrive At Night?

A couple of weeks ago I presented at Qbit's TestExpo in Manchester.  Rare indeed is it for me to go there these days.  There was a good turnout, a whole bunch of presenters and me to wrap up the show before the prizes, the bar and the night shift.  What would you expect for an audience of testers? Having spent the day gabbing I was sure they'd have to go back to work right after I was done.

These are the slides from the presentation I gave (If you're reading this in an RSS reader such as Google Reader, you're probably not going to be able to see the slides so click on the blog title to get to the page itself and you'll be able to see them.  Alternatively, if you really don't want to visit the blog, you can access them at this page on


The essence of the show was a proposition that too often in programmes, testing is treated as contingency time.  Because testing can't start until design and build are broadly completed, the only scope that ever gets cut is that of test.  Programmes are generally loath to over-run and, even if they do, it's rarely because more time is needed for testing. So, to get as much in as possible, all the stops come out and long nights and many weekends are worked over and over to try and hit the live date.  Inevitably stuff gets missed.

Every so often over the next few weeks I'll pull one of the slides out and post it here as a picture and talk through the things that I said at Qbit - very roughly at least, I have no script (how did Cameron put it?  "It'll be a bit of a mess but it will be me"?) so whatever I post will doubtless not match what I said and here, of course, I get more room to be lazy and expand my points.

Tuesday, October 02, 2007

The Only iPhone Review You'll Ever Need - Part 3

Things that make you go "uhhh..."  I love most of what the iPhone does but there are some things that just make you slap your hand to your forehead.  Some of these can, I hope, be fixed with future software releases, some will doubtless not be fixed until the hardware changes. Some will never be fixed.

First, the things that everyone, especially the UK press, have already complained heavily about:

- GPRS/Edge. It's slow, yadda yadda. Every pundit has slammed Apple for using this 2.5G or even 2G network and for being dumb enough to launch so a laughably slow phone in the European market that is oh so saturated with 3G service after the operators spent their [shareholders] billions (some stats here and here - my reading says 3G market share is about 20% of total in handsets but that there's limited use, certainly very few browsing the 'net or doing email).    The difference between "good" edge and standard 3G is not that great.  The difference between full HSDPA and edge is enormous. So Apple has certainly fallen behind the very best but is not far behind most phones.  Blackberries invariably work on 2G, mostly GPRS and not even edge - but their background send/receive makes that less of an issue. The trade-off, though, is plainly increased battery life, a bigger screen, and ready access to Wifi (via the Cloud once the iPhone launches in the UK). So far I've really not seen the issue - if you're working on all you can eat data tariff (fair usage rules 'n' all) and have occasional access to Wifi, you're probably just fine. But, if you're running Wifi, bluetooth and video a lot you'll probably see less than a day of battery life - with 3G too it might have been half that. My guess is that, apart from the PR and marketing hype from the other vendors and the noise in the press, you won't much care about this "downgrade" to an older network)

- The camera. It's "adequate" at best, it doesn't do video and don't expect to send any MMS.  The review I link to does a far better explanation of what it's like.  I've taken a few pictures with the camera and it seems ok or, at least, on a par with the Treo (which had an awful camera) but not as good as the HTC Touch (which had a range of options allowing photos to be adjusted for indoors and outdoors).  I take "fun" snaps with my phone, not pictures I want to print or put on a slide show.  Sometimes I use those photos in my presentation slides and they work fine for that.  But this is not a "camera phone", it's a "phone camera".  In fact it's a "Phone iPod camera."

So I'm not sweating either of these.  I know Apple will come out with a new phone next year that upgrades the hardware and I'm pretty sure that some of this stuff will be improved in some software releases over the coming months.  I'm also hoping that the iPhone might actually be a catalyst for more websites to do mobile versions of their websites - and for all of them to auto-detect that you're viewing them on an iPhone.  We built a mobile version for years ago (that's not quite true - we built style sheets for small screens and auto-detected when a mobile browser was being used so that the pictures were stripped and the navigation was made simpler; it's not hugely difficult).  It's actually very frustrating to use google's mobile search, click a link and then be thrown straight into a page with flash, big graphics, left and right hand navigation and so on.  If the iPhone gives folks the nod that it's ok to use small screens, all the better for everyone.

But the stuff that doesn't quite gel for me ...

- SMS sending. Texts are sent "while you wait" and sometimes that feels like a long wait.  Whilst sending there's a white bar that fills from 0 to 100% (or perhaps 0 to 168 characters, who knows?).  You can't do anything whilst that's happening.  I'm used to phones - Treos and even Windows Mobile - that send in the background and flag an error if there was a problem sending.  You're also unable to forward one text to another person (see also the "cursor" point below"). This, of course, means that you can't send a text while you have no reception, say when you're on the tube, and expect it to send as soon as you have signal again.  You also can't forward a text or send it to multiple recipients from within a chat stream.

- SMS opening.  When you switch to SMS the iPhone opens the SMS application wherever you last were.  Even if you have new texts and you were in the middle of a thread when you last used it, it opens back in that thread.  The odds are, if I have new texts, I want to look at those.  On the upside, if you're watching a video, it will flash the text up on the screen with a view/ignore dialogue.  I like that.

- The cursor.  Simply put, there isn't one.  If you make an error in some text and need to go back to edit that text, you'll have to poke at the screen with your finger until you get to the right place.  I doubt that will be with the first poke, it certainly isn't with me and I don't think I've got particularly fat fingers.  By the same token, don't expect to be able to copy text, drag and drop or cut and paste.  I have no idea how this gets remedied given there are no buttons to speak of (and i don't want any more than those that exist now) - it may be as simple as a menu bar at the bottom of text with left/right/select, inelegant though that is (and therefore probably why it's not there now)

- Applications.  Apple has spent years evangelising its stuff, creating a tight knit community of developers who have supported it through thick and thin.  With recent operating systems they've created not just applications but widgets and gadgets.  The operating system is a platform and other people fill it with things that they think customers want; if Apple comes out with a better product, then the market adopts Apple's product, if they don't come out with a product then someone else fills the niche and the customers are happy.  This approach has worked just as well for Palm (with 1000s of applications) and, of course, Microsoft who kicked Apple's butt (and everyone else's along the way) by opening up (we can argue about degrees) their operating system and working harder than anyone else to get developer support.  Now I'm expecting that Apple's stance is temporary - after all, plainly the iPhone is not even close to being "finished" yet - whilst they mess with version 1 of the iPhone and that as they stabilise it and stop making big changes, they'll allow application deployment right to the phone.  Sure it might mean several pages for the Home Page or it might mean folders for storing things.  It might also be niche - some people will just like their iPhone pristine - and I suspect that this is the bulk of the market; folks don't want their phones like their PCs, bloated and prone to crashing as the PC sceptics would say.  But lots of people will want to do other things with it from figuring out where they are in rudimentary GPS terms to waving the phone around like a Wii controller playing pong.  Who knows? Who cares? They should be allowed to do it.  Personally, I want a version of Robert Parker's wine database and a Nintendo Brain Training programme (the second to help recover from the damage of the first).

- Headphones. What were you thinking Apple?  You straight away disabled my shiny Shure headphones by making the socket so different on the iPhone that they can't plug in.  Why would you do that?  You don't even sell an adapter in the store - I have to go and get one somewhere else.  Your headphones are average at best - indeed more sound seems to leak from them than go into my ear.

As with previous posts, I'll update this as I find more. 

Sunday, September 30, 2007

The Only iPhone Review You'll Ever Need - Part 2

I said I'd talk about the good things about the iPhone first - the things that make you go Hmmmm. But first, some context, especially for those who think I'm a gadget obsesses, chaser of trivialities (you're not wrong, it's just there's a reason as well).  I've had my problems with phones, and have often posted ... e.g. about the SamsungTreo 600, all Nokia phones, the Sony Ericsson range at large, early 3g phones, or just phones in general. I could have posted more previous references; the point is that I don't think much of mobile phones: they frequently crash, have trouble synchronising, have battery problems, aren't obvious or are just plain useless.  Been there, screen printed several of my own t-shirts.

My criteria for acceptance have been, from past posts:

- Has a battery that lasts at least a long weekend (treo 750 need not apply)
- Has a keyboard that registers your touch simply and effectively (samsung sgh600i need not apply)
- Doesn't crash more than once a week
- Works consistently.
- Adheres to standards
- Allows me to synchronise text messages, ring tones, speed dials to the PC
So, what about the iPhone?  Well, here are the things that make it stand out, for me at least:

- The screen.  It's a big call Apple are asking you to make: big screen/no keyboard versus smaller screen/keyboard. Corporate warriors looking at their blackberries should absolutely stay with those; but individuals who make calls and do lots of text, I think you have a real choice for the first time. I've played with many keyboardless phones (P8xx and P9xx had keyboard but were just useless with them so I took the keyboards off with the handy little screwdriver that was provided; HTC touch was better but not great).  The screen real estate is brilliant. It not only lets you play with all the ipod stuff - video, coverflow and so on - but it actually gives you room to see, say, a whole contact on one screen, or several similar contacts in a list, or a decent sized wedge of google maps when you're lost in Manchester (as I was last week - when I used the iPhone to show my taxi driver where the hotel that I was staying at was).

- The keyboard.  I was a sceptic, I admit it.  Even in the Apple store I wasn't convinced. After a few days use at home I still wasn't convinced.  And then it dawned on me - you have to trust the system.  Windows Mobile, as you peck at the tiny keys with a stylus (fat fingers no use), tries to guess what you're going to type and tries to save you time by giving you 2 or 3 choices.  Apple's system guesses what you really meant to type - dealing with horrible mis-types at the same time.  And, more often that not, it gets it right.  Typing texts becomes a one finger flier.   It's not touch-typing, it's not even feedback typing (I have sound off) but it's staggeringly fast.  You still have to watch for some words - if you type "si" instead of "so", the iPhone continues as if nothing happened for instance. The HTC touch, whilst a great phone, actually stopped me sending texts - it was so painfully difficult that the effort was worse than the reward; I just stopped sending texts.  The iPhone has converted me.  And, it's a fair trade between screen and keyboard.  I loved my Treos - 600, 680, 750 - for the keyboards and for one single innovation that they alone had, until now:

- Threaded text. I don't get why every phone doesn't have this.  It's essential.  All your conversations grouped together under the name of the person you're texting.  If you often have 5 or 6 text conversations going on and can't quite remember what you said to who sometimes, this will sort that out right away.

- It's the software stupid. Rarely have I seen a software upgrade applied to a mobile phone during the time that I owned it.  Once or twice the Treo folks published an upgrade but, more often that not, they kept it to themselves or just released it to new phones, not as an upgradeable item. Even the recent Windows Mobile 6 upgrade for the 750 has not really made it to the street (pirate Australian versions not included), despite it being more stable and with far, far better battery life than WM5. Apple have already released software upgrades just a couple of months after launch. And they're following a different strategy from Sony's with, say, the PSP.  They're actually offering you things you might want in return for things that they want - for instance, for the first time that I can ever remember, your bluetooth headset battery life will now show on the phone's main screen. They're also correcting the "duh!" functions that they should have had on day one - tapping 2 spaces will now insert a full stop a la blackberry.  In return, they're locking down some security holes that the hackers have exploited.   I have no doubt the hackers will get round it, but it encourages people to go legitimate for sure - show me a hacker who will volunteer to upgrade his/her phone first with all the recent problems around 1.1.1.  That said, what should have been a beautiful moment for Apple, the ability to add functionality on the fly, is going to be mostly ruined by adverse PR (and even lawsuits, although there's nothing new in that for most companies these days) as the upgrade apparently renders hacked 'phones useless - no need for that Apple, just don't let the install go ahead, detecting before install that there's something wrong.  Even Microsoft didn't ever shutdown a PC with a pirate copy of Windows.

- Three words: "slide to unlock". No more pocket noise.  With a name beginning with "A" (or "a" to some) I'm often the unintended recipient of calls or texts, sometimes a dozen in a day from the same person (only 1 or 2 persistently do this; you can imagine my response).  A very few times I've learned positively secret and commercially sensitive information as I listened in, and walked in to the room where the conversation was taking place holding my phone out so that they'd know what they'd done. You just can't make rogue calls with this phone.

- Proximity sensor.  No more beeps and squelches as you press buttons on the phone whilst it's held up to your ear.  The Treo 750v used to spend most of the time during a call changing the time or the alarm function, even though the screen was locked - it just didn't seem able to resist. Other phones have been much worse.  The iPhone disables the screen when it's close to your ear and enables it again when you lower the phone.

- Big icons, no menus.  Press the single button, get the home screen up, select what you want to do and just go.  No more pressing of menu buttons, searching up and down for the menu you want, selecting the sub-option and starting what you'd planned.  The bulk of the things that you want to do are right there on the home screen or a layer below. Want to check stocks? one button.  The weather? One button. Text? One button. Browse the web? One button.

- Rubber band scrolling.  This may be a more binary thing - people will hate it or love it.  I have just over 1,000 contacts in my iPhone.  On earlier phones I've always dialled someone by typing the first few characters of their name and then selecting it. The iPhone's rubber band scrolling was born to negate this - select the start letter and then scroll up or down at speed - see all the numbers in one go and then tap the one you want.  Send a text with one tap, add it to favourites to make it easier to get next time.

- Battery life.  I'm getting 2 clear days with a mixture of phones, text, wifi, GPRS, video and audio books.  It's good, not great.  But it's better than any number of Windows Mobile 5 devices and as good as any WM6 device (although on none of those did I ever watch video, play audio or use wifi)

- Rock steady.  Not a single crash or frozen screen so far. No spinning hourglass or multi-coloured kaleidoscope indicator.  No hang-ups.  No missed calls because it didn't ring. No reboots. Just rock steady.

- Bluetooth.  I've never been a fan of bluetooth.  That whole set to discovery, discover, type the 4 digit passcode stuff just gets my goat (no, not the same directionless goat).  The Apple bluetooth headset comes with a new cradle for the phone - the headset goes in its own slot and auto-pairs with the phone.  No buttons to press, no passwords to guess.  On top of that, when you turn the headset on the iPhone automatically switches bluetooth on and connects to the headset; when you turn it off, the iPhone turns bluetooth off.  Sure, it's not stereo for music and sure it's still bluetooth with its variable reception quality, but it's hassle free bluetooth.

- Synchronisation.  I keep my calendar on a PC in outlook, my music on a Mac (via a NAS).  I honestly expected some trouble synchronising to two devices and certainly syncing to a PC.  There were no problems.  Contacts and calendars moved straight away without a hitch and without any software install; music synced up nicely as did video.  The sync for video is a little odd - for TV shows it wants you to sync the X last unwatched, rather than perhaps the first Y unwatched.  But it just works.  Motorola phones have, in the past, required exports to CSV, edits and re-imports and other phones have just not synced properly. iTunes also backs up items such as SMS allowing you to restore them if there are problems - the only phones I've ever seen do that are the Palm-based Treos.

To follow this, in the next few days, I'll publish the things that make you go "uhhhh", the things that aren't so great, some of which I'm sure they'll fix in software and some that they won't. I may even add to this post too over the next couple of days to round it out.