Saturday, November 24, 2007

Do Senior Civil Servants understand data?

As a database developer and it is with disbelief that I watched the disc story unfold. The story about the two CD's sent by post that were lost and the office messenger boy who does the carrying things to the mail room was blamed for losing 25 million sets of personal data.

1. There were 2 CD's. This means that the data was at most 1.5Gb in size. I dont know what speed the datalink at the Revenue office is but lets assume it is a bog standard small office link. The download speed could be 20Mb a second, ie 5000 seconds per Gb.(less then an hour)
The upload speed would be a 8 times slower.

Secure ftp could have been used to acheive the next day delivery that TNT was offering.
I think one will find that they have a super fast uncontended digital connection which would have much higher upload and download speeds.

2. As for the cost associated with filtering this data, It would take me 5 mins to construct a filter - no matter what database format was used, incl. ones I am not familiar with. I would say that there is probably at least 50,000 people in the UK with these particular skills. By the way my wifes laptop could run this filter on 1.5Gb of data in less then 10 secs.

3. It is hard to imagine that the person in charge of and controlling this data does not have these database skills, or is assisted by someone with the skills.

4. The question of whether someone followed departmental procedure is a red herring to avoid the real question, which is about how the procedure was arrived at, in the first place.

5. If the people controlling these databases are so backward in their understanding of data and data manipulation, how could they ever be remotely trusted to be able to control the integrity of the data they have and the forthcoming ID card databases. The database technician has more control of it then they do.

6. I would bet the password protection on the disc is a widely available proprietary one that could be bypassed
by a $50 download.

No comments: