Castlebridge Associates

One of the most popular presenters on one of the most popular radio stations in Ireland recently launched a great idea – a loyalty card for his listeners. This card seems to be the replacement for his previous gimmick, a “Nudie Pen”.

Visit the radio station website (NewsTalk.ie, tell them your name, your address, your email address, your 3 favourite bands and your favourite foods and a piece of plastic featuring a picture of the host will wend its way to your door.

Simple.

At least it is unless you step back and think about the process from the point of view of Data Protection principles.

Personal data must be obtained and processed fairly for specific purposes. What are the purposes for which NewsTalk wants my personal data? If it is just to send me a card then we walk right into another issue – information gathered should not be excessive to that purpose.

So, if you are just sending me a card, why do you need to know my music and food preferences?

Sensitive personal data, such as data pertaining to medical conditions or political beliefs or ethnic origins is treated with more seriousness under the Data Protection Act. So, depending on the responses to those questions about music and favourite foods, sensitive personal data could be being processed.

The explanation of the loyalty card scheme that is on the NewsTalk website is great and in keeping with the light hearted nature of Tom’s show. However it doesn’t go far enough in explaining or setting out the purposes for which the data is being captured.

Other issues arise as a result of processing personal data via a website, such as the legal requirement to have a privacy policy displayed on the site and the data protection requirements of keeping the data safe and secure and only keeping it for as long as it is needed for the specified purpose. I’ll explore these in later posts.

It is all too easy to fall foul of the simple rules that exist to ensure trust and transparency in how personal data can be processed. Prior planning can ensure that Compliance is an enabler of business and customer interaction rather than a nagging fear of being caught dragging at your actions.

Taking out your Nudie Pen and mapping out what your information objectives, purposes, etc. are (see this tutorial on my company website for an example) is time well spent to make sure you aren’t creating a rod to beat yourself with. Using your Nudie pen to sign up for some Data Protection Training (such as that offered by the Irish Computer Society or my company) would also be a worthwhile step, particularly given the Data Protection Commissioner’s recent findings on the need for the management teams in businesses to be aware of the Data Protection implications of their actions.

A few weeks back I got a new iphone. I’d resisted for years, enjoying the pleasures of Nokia and Symbian and the challenges of Palm and Windows Mobile 6.1.

The fun part for me of any new mobile phone purchase is playing with the new toy  tool and seeing what it can do that my old one couldn’t. For example, back in the 1990s when I did my first upgrade from my first mobile phone (an ericsson model so old that I actually can’t find it referenced on the internet), I found that the new phone was so much smaller and lighter I was actually able to carry it around.

The irritation I have is when it comes to moving my contacts and synchronising with my various other technologies that hold contact details (laptop, gmail, company address book). Inevitably I wind up with duplication and triplication of contacts. I thought I had the problem licked on the iphone though as there are a number of apps available for managing contact details and reducing duplicates.

However, having spent a few days using them I am unimpressed as they seem to be making a the traditional rookie mistake in de-duping records – assuming that name matching is enough.

My brother and father share a given name and a family name. They have different middle initials, different addresses, different phone numbers, different email addresses (all the stuff that you would have in a contact record on your phone). Each application I tried decided that they were a duplicate entry and merged the records. This was annoying.

In other cases, I have duplicate entries with varying degrees of record completeness. For example, my friend Cathal exists at least 4 times, with one entry having most of his contact details,  with spurious email addresses or social networking nicknames in the others.  The “data quality tool” very kindly merged all the records into the entry that had the least amount of data, and deleting the other records.

Right now I’m considering firing up talend, datanomic, or informatica tools to dedupe a dump from my iphone and reload it to the phone, and then hopefully that will cascade through the rest of my data stores when I synchronise.

But I’ll need to draw a data flow map of all of that to make sure.

Grrrrhhh.

So. If the existing tools for data quality on the iphone are not up to the jobs, what is missing? The good news is that the data sets are fairly clearly structured (once they get into the iphone), so that is less of a concern than the actual processing of matching and consolidation of records.

1. Probability scoring across multiple fields would be nice. If two people have the same name but significantly different contact details then it is very probable they are not the same person. A corollary – if there are two records with the same name and one has contact information and the other record has only a name, chances are they are duplicates.
2. Presentation of matches for review. While the machine can make good guesses where the name and contact details are the same, where there is confusion, the matches should be flagged for a review by the phone user (the “Data Controller”). This way we can avoid having to unpick erroneous matches.
3. Merging of records should be done on a more structured basis, with mapping of fields being user-customisable based on a standard template. I despair of important contact information being dumped into a notes field (it reminds me too much of when I had to try and migrate data out of a Siebel call centre system a few years ago).
4. The matching should be able to cater for multi-lingual input (as phones don’t all live and work in english speaking lands).

There may be other requirements that I am not thinking of here at the moment, but those 4 are a starting point. Perhaps an obliging Data Quality tool vendor will develop an iphone app to a web service for matching contact records.

Personally, I think that having such a service available would help raise awareness of the value of quality non-duplicated contact information to individuals and to organisations.  However, the app on its own isn’t enough as the average smart-phone user may have personal information held in a variety of places and, just like in a large enterprise with lots of data stores, creating a “Single View of Contact” will require you to understand the flow of your contact information around your tools (i.e. does the phone update the laptop and does the laptop synch to google apps and does google apps synch to the phone?) to avoid the cleanup work being undone the next time you plug your phone into your PC.

Information Quality Management poses challenges for the enterprise, but can also create friction for the individual trying to manage something as simple as a list of contacts across multiple information stores.

Do we have an app for that?

In a taste of the change of emphasis that is seeping through the global financial services industry, the Irish Financial Services Regulatory Authority is pursuing 24 cases of overcharging by banks and insurance companies, according to this morning’s Irish Independent Of course, stories of financial services overcharging and other information quality disasters in that industry are [...]

Yesterday I took some time out from work to help hang some new light fittings at home. Our local handyman/neighbour was doing the hard work as my wife has seen enough of my father’s DIY exploits to have put an embargo on me even looking sideways at power tools.

The estimated duration of the job was to be about 45 minutes to an hour to hang three fittings. The first two fittings went up in about 20 minutes. The final one, that took us about 4 hours (and as of this morning still isn’t finished. We hadn’t factored on the “creativity” of the electricians who installed the original wiring.

When we opened up the existing light fitting in the living room we were faced with a spaghetti junction of cables. When we wired them into the new light fitting, the light went on but the switch wasn’t controlling it. It seemed we’d wired the light into a loop going somewhere else. We were faced with 5 live wires which had been going into 4 connectors on a connector block. So we had to then test each of the possible live/neutral combinations in turn to find the ones that actually related to the switch (which necessitated our handyman/neighbour having to play with live 240 volt electricity, which is never a good idea).

When we traced the correct cable pair I did a very simple thing. I dug out my label maker and put a label on the cables that related to the lighting circuit in that room. It struck me that that 30 seconds of effort was something that the electrician who wired the house could have easily done when they were installing the cables, making life simpler for him (or her) and for anyone who came after.

We wired everything up and fitted it up for a quick test before finishing the job. I turned the power back on.

Then there was a loud bang and the power went out.

It turned out that there was a break in the live wire we’d just labelled (the important one for the task at hand) slightly further up the cable from where the label was which had pierced through the insulation and come into contact with the metal mounting plate for the light fitting.

As a result, the magic smoke had escaped from the circuit breaker and the light switch.

What had ensued for my neighbourhood handyman and I was instead frustration as  a task which should have taken a half hour stretching into nearly six hours (over 2 days) and additional expense (to the handyman) in replacing the blown components.

To put it another way, for the want of €0.15 of labelling on the part of the original vendor to identify the attributes of the various wires we found (such as “this one runs the lights”), I expended a full half-day of work and the handyman was unavailable for other jobs which would have paid him a lot more than the rate we’d struck for fitting the lights – and that was before the additional cost and complication of having to go to the electrical wholesalers this morning to buy replacement parts and fit them as well.

It struck me that this is a situation we encounter on a regular basis with the information assets of an organisation.

Very often the important data for a given process in a given area is not clearly identified. Management say “give us everything and we’ll figure it out” and call centre screens and web-forms are cluttered with a variety of information capture points.

A failure to understand (or label) the purpose of that information, where it comes from and where it goes to, and its critical path in the business can result in undesired outcomes as soon as anything starts to change in the business, business processes, or technology platform (such as replacing your front end systems with a new one, the nearest analogy I can think of for changing a light fitting).

This results in expended effort on scrap and rework trying to get the blasted thing to work right with the desired outcomes (such as throwing illumination on a problem), and quite often can result in a critical information path way being blown and needing replacement or an internal control process in the business stopping a process.

Of course, things can often be worse in the Information Quality space where the internal controls on quality may not function as efficiently as a circuit breaker and a light switch which have planned failure built in to them to isolate the end user from the dangers of domestic electricity supply. When controls like circuit breakers fail, the results can be… shocking.

Sometimes it is the simplest things that are important, such as knowing what wires relate to the circuit you are fitting a light into, or what items of information are actually critical to the success or failure of a process (both the immediate process and down stream -remember  there were 4 other live wires relating to other circuits that had to be dealt with as well) is a key contributor to the success or failure of any change effort.

What controls do you have to protect your business knowledge workers from the dangers of a high voltage low quality information? Are the mission critical data in your organisation clearly labelled?

From The Register comes this wonderful example of the problems that can arise where data is used for unintended purposes, resulting in poor quality outcomes for all involved. The NYPD have been regularly raiding the home of an elderly Brooklyn couple. They’ve been hit 50 times over the past 4 years, which might mark them out [...]

Courtesy of Dataqualitypro.com on twitter comes this story about how Virgin Media in the UK have sent a letter to a customer prospect addressing them as “Mr Illegal Immigrant”. Virgin Media have investigated the issue and state that it was an isolated incident arising from data purchased from a 3rd party. This suggests that Virgin Media [...]

I found this on http://www.motivatedphotos.com and it struck me that it is a wonderful metaphor for data integration, information quality, and data governance in many organisations where they are reacting to issues, sustaining silos, or viewing all of this as an IT issue rather than a business challenge, or trying to solve the challenge with series of fragmented department level initiatives.

Thoughts?

From today’s SUN tabloid newspaper in the UK comes this great story about how automated processes without adequate business rules and checks on information can cause embarrassment. It seems that local government authorities in the UK have been fining themselves for parking offences and then suing themselves when the fines aren’t paid. Things have even gone [...]