I am pleased to announce the publication of 101 Lightbulb Moments in Data Management. The book features contributions from some of the most respected minds in data management, including Jim Harris, Jill Dyché, David Loshin, Phil Simon, and others. The book represents the best of The DataRoundtable by Dataflux. DataFlux CEO Tony Fisher has written the Foreword.

The book is now available on Amazon.com.

This is post sponsored by the Enterprise CIO Forum and HP.

My friend Vinnie is considering buying a MacBook Air. His current laptop has given him years of productive use but, as with all things technological, eventually you have to upgrade. Vinnie knows that he should buy “too much” computer. That is, he should go heavy on the RAM and make sure that his hard drive has oodles (my word) of space. Better to be ahead of the curve.

A new and more powerful computer will certainly increase the speed at which he can surf the net. He’ll have a more current operating system. Vinnie will be able to load applications and use more of them concurrently–although in today’s browser-based world many erstwhile computer programs have been effectively replaced with new tabs in a browser.

Whether Vinnie buys a Mac or a new PC with the highly praised Windows 7, Vinnie’s life will become easier after he learns the new operating system and loads the applications that he needs.

But let’s talk about what his new computer won’t do:

• A Mac won’t “de-dupe” the records in his customer spreadsheet.
• No PC is going to analyze the records in his accounting package. No will it identify questionable vendor records and accounts.
• Since Vinnie uses multiple sites to manage his projects, no computer will magically merge largely unstructured data from each.

So, what does Vinnie’s dilemma have to teach us about the corporate world?

Data and the Limits to Application Modernization

In a post entitled “Are legacy applications causing your recovery to drag?“, my friend Charlie Bess of HP writes:

The slow recovery requires you to have more flexible and integrated systems–after all technology, like customers, didn’t stop advancing during the downturn. There are also new automation techniques available that increase the quality while lowering costs. No users will stand for a hodgepodge of solutions thrown together into a virtual application scrum–users will want to see the interface, the data and the process integrated into a more intuitive and cohesive interaction. This takes enterprise architecture, process and interface design. Not something that can be done overnight.

Bess is absolutely right here. Organizations need to embrace contemporary applications and platforms to take the next step. But there are significant limits to what application modernization can do.

For example, let’s say that your organization is currently running its CRM system on a clunky mainframe. No one can access this system–and its data–without going through multiple steps and VPNs. The data in the system is a mess.

So, you’re thinking of moving to a SaaS CRM application. Will you have a more contemporary system? Yes. Will you be able to get at your information more easily? Yes. Will your data still be a mess? Yes.

Simon Says

It’s easy to blame bad data on legacy apps. Don’t make that mistake. If you improve the apps but leave the data alone, have you actually solved any problems? Or do you just have fancier screens still populated with bad data?

Feedback

What say you?

This is post sponsored by the Enterprise CIO Forum and HP.

Here’s an excerpt:

Data Quality Pro: Bloor Research uncovered some interesting findings in their industry survey recently. They found that the failure rate was extremely high. What do you feel is preventing these companies learning from their mistakes?

Phil Simon: In a word, experience. Many of my clients have never been through a major new system implementation before. Most have never replaced their legacy back office systems with one integrated solution, at least with current staff members and levels. Such a project is obviously much more time-consuming than a minor upgrade of a stand-alone application that isn’t really connected to anything else.

To read the entire interview, click here.

I will be appearing on DM Radio on Thursday, Feb. 4 at 3 pm EST to talk about data integrity.

Description

Getting the big picture with data quality requires a whole host of sound procedures, starting with data lineage, and extending through cleansing, transformation, delivery, and usage. Altogether, these processes encompass a sound data governance strategy.  Failure at any control point can result in serious errors that can undermine a organization’s fundamental operations.

I am one of several guest on the show. You can listen to it clicking here.

• facilitate document sharing and updating
• improve intra-team communication
• rid the organization of its dependency on email

Collectively, these collaborative tools can improve project management. Through their successful and consistent usage, the organization can improve its chances of achieving its goal: a smoothly run project.

Sounds good in theory, right? Unfortunately, many times the promise of these tools is far greater than their actual benefits. (This is also true of Enterprise 2.0 in general.)

But why is this the case?  I’d offer two simple and related reasons:

• Collaborative tools are only as good as the people who use them
• Old habits (read: email) die hard

Email may be the Internet’s first killer app, as project-related messages convey important information about tasks, dates, and news. From a collaborative standpoint, there are a few major problems with email:

• Emails can easily make the content on a wiki dated or even irrelevant
• Emails tend to be much less easier to find and search than content posted on wikis
• People constantly forget to copy others on email
• Most emails are downloaded to individual PCs, making them suboptimal for future reference

To be certain, wikis will never obviate the need for emails. What’s more, not every piece of information on a project should be posted on a wiki. “Hey Vince, didn’t you think that Nikki sounded like an idiot during the meeting today?” However, emails should constantly reference collaborative sites to reinforce the notion that the wiki governs the project, not 100 disparate emails.

Solutions

Organizations should take the following steps to ensure the optimal use of collaborative tools:

Start at the top

The PM or project leader sets the tone for the entire group. It’s hard to expect individual end-users to move from emails to wikis when the PM doesn’t lead by example. This type of leadership also includes making gentle suggestions to those who rely on email or—perish the thought—don’t update their documents anywhere.

Hold team members accountable for updates on wikis

Individual end-users must use collaborative tools consistently throughout the project. This goes beyond updating their own availability or progress. If the organization uses SharePoint, for example, then it needs to be the epicenter of the project. Unless the material is confidential or politically sensitive, all project plans, test scripts, requirements, and training materials need to go on the wiki. Period.

Cross-reference wikis in emails

Today, many people now routinely read email via BlackBerrys, iPhones, and other mobile devices. This isn’t changing anytime soon. To that extent, expecting everyone to abandon email simply is folly. However, emails should contain URLs to the same content on wikis. Doing so will help minimize conflicting or missing information.

“We are not done,” hinting that acquisitions could be forthcoming. But even Monster’s architects see the writing on the wall. Bill Warren, the founder of an early job board that morphed into Monster, is now executive director of the Direct Employers Assn., a consortium of corporate employers. He’s partnering with the owner of the “.jobs” domain and will launch job sites under that domain later this year. Says Warren: “The days of the big, expensive job boards are over.”

Wow.  How the time flies. Didn’t monster.com and its ilk just replace newspapers a short while ago?

Other questions popped into my mind:

• What’s going to replace LinkedIn or, more broadly, social networking sites?
• Is any technology here to stay?
• Is monster.com grasping for salvation? (Yes, I just read Jim Collins’ How the Mighty Fall.)

Particularly interesting is LinkedIn’s use of push technology:

Push technology, or server push, describes a style of Internet-based communication where the request for a given transaction is initiated by the publisher or central server. It is contrasted with pull technology, where the request for the transmission of information is initiated by the receiver or client. (Source: Wikipedia)

As it relates to job searches, push technology allows recruiters and hiring managers to receive the profiles of job candidates without having to actively “pull” them via searches. If I’m searching for a new IT Manager, for example, LinkedIn will find and send me the profiles of qualified candidates without my having to wade through the backgrounds of hundreds of potentially underqualified applicants. This saves me time and makes the entire process less costly and cumbersome. While it cannot guarantee a good hire, I’d love to see metrics assessing that very question. I suppose that that’s a question for business intelligence gurus.

I suppose that my main point is that no organization–not even a successful one founded during the dot com boom–can rest on its laurels. New technologies can be used quickly to displace industry leaders.

Reading Brian Sommer’s recent post on client incompetence made me think of a critical point on software implementations: clients and consultants often have unique and different focuses during business hours.

The Consultant

As a consultant, my focus at a client site is singular when I am engaged to implement a new system, write reports, train a course, etc. When on site to implement a new system, I usually don’t have a “day job” that would divert my attention in the event of a problem with a client’s current applications. Even if I did, many times I wouldn’t know where to start. Issues such as these are often well beyond my scope or ability to handle:

• I don’t know how to resolve an accounting issue in a client’s legacy system because I have never worked with it before.
• I can’t extract data from a mainframe because the extraction tools are antiquated and I have no experience with them.

Let’s assume that I do have the knowledge of a client’s legacy systems and can pitch in. Let’s also assume that the organization is comfortable the “all hands on deck” approach. From an insurance and liability perspective, however, I may not be able to do this for two reasons:

• The statement of work typically does not cover my working on applications with which I am not familiar.
• From a Sarbanes-Oxley perspective, I may not legally be able to get involved.

In the end, for the consultant willing and able are often not the same.

The Client

Clients, on the other hand, are constantly balancing (or trying to balance) present and future priorities. The former almost always defeat the latter, causing

• Project delays
• Cost overruns
• Critical oversights
• Minimized knowledge transfer

This problem is particularly acute at understaffed organizations. Heaven forbid that one of the following scenarios occurs:

• An organization loses a key contributor unexpectedly.
• That end-user’s responsibilities are not (sufficiently) documented, much less understood.
• The organization is confronted with an emergency requiring the immediate and undivided attention of current end-users.
• The project has a “hard stop” for budgetary and/or date reasons.

The end result is that the organization is more susceptible to major problems, oversights, and project failures.

Enterprise Systems, Staff Reductions, and Employee Training
Systems do not exist in a vacuum. A “state of the art” system run by too few or poorly-trained employees poses great risk to organizations. No HRIS or payroll system can possibly catch every type of mistake. The current economic climate exacerbates this risk. With respect to headcount, layoffs increase the vicious cycle of risk to organizations:

• Organizations have an incentive to trim staff and reduce—if not eliminate—formal training and opportunities for end-users to learn.
• This solidifies many end-users’ bad habits and suboptimal processing methods.
• Many end-users’ responsibilities have increased significantly
• In the event of layoffs, more work among fewer employees means even less time for “cross pollination.”

Management should be careful when cutting “non-essential” employees, as they can quickly become essential. For example, an organization has four HR clerks to process paperwork. While no one clerk is absolutely essential, reducing that number to two now changes that equation. If staff reductions are truly necessary, organizations must ensure that departing employees’ daily responsibilities are both sufficiently documented and well-understood by others in the organization before they leave.

Organizations need to identify essential employees via succession planning. Which individuals can the organization not afford to lose? It is imperative that they are proactive; they should attempt to anticipate any key employee defections.

On the training front, organizations should strongly consider cross-training end-users in multiple functions. Two super users with substantial skills and a global perspective may be able to do the work of three or four limited end-users, especially if they are skilled in different automation methods. For example, consider Mary, an end-user who is very skilled at Microsoft Excel. Her organization has a Crystal Reports license, but no one really uses it. Sending her to a class would allow the organization to finally realize the benefits of Crystal; no longer would reports have to be cobbled together manually.

Read the rest of the article by clicking here.

Egolock–when an exec makes a bad decision, refuses to admit it change, and holds steadfast until the bitter end.

While the term is new to me, sadly, my experience with this condition is not. This post focuses on egolock in relation to enterprise systems.

Most Common Occurrences

In my experience, egolock is more likely to be found in:

• Large, mature organizations – Management at start-ups and relatively new organizations is less likely to “stick to their guns” at more agile companies.  I am reminded of a consulting project on which I explained to a senior manager at a medium-sized VOIP company that its PHP-based systems might not be sufficient to support its long-term growth objectives. Much to my surprise, that senior manager said something to the effect of, “I thought so. No worries.” I was floored. I did not expect such a consultant-friend response. Although he had spent years developing these proprietary systems, he understood the limitations of his “babies” and opted to get on board with a more robust enterprise solution.
• Organizations with more mature management - Executives more familiar with traditional software development and implementation cycles often fail to grasp with new world of SOA, web services, and open source. In the 70s and 80s, many CXOs of today were cutting their teeth. Despite what they may have heard about some of these new terms, they’re uncertain about translating them into realities at their organizations. They know what works–or at least what has traditionally worked.  It’s often too much of a departure for them to grasp the present and future, even if their existing systems have significant limitations. Call it fear of the unknown.
• Industries with high profit margins – Executives in government and healthcare (two industries with which I have had a great deal of experience) tend not to look at the world through the same lenses as their brethren in more competitive industries. In telecom or retail, a bad decision–or refusal to act–may result in a company failing. Period. Most hospitals, government agencies, and non-profits do not face imminent threats of extinction. Folks in these corner offices can afford to stick to their guns; their organizations are probably not going out of business anytime soon.

One Potential Solution

I have long advocated that consultants act as true business partners–i.e., challenging poor client decisions and not just rubber-stamping suboptimal system configurations. Of course, executives with the most severe cases of egolock cannot be reached. They will either choose a compliant consultancy from the get-go or replace a “difficult” partner if it tries to steer the organization towards a more practical solution. Lamentably, sometimes obstinate senior managers only realize the errors of their ways after a system or project has crashed and burned. While they may fall on their swords, they always have one convenient scapegoat: the consultant!