Posts Tagged ‘compliance’

Data Quality: You Don’t Know What You Don’t Know

January 19th, 2010 3 comments

SAP BusinessObjects delivers great Business Intelligence solutions so that organizations can report off their existing data sources.  But what is the point of reporting of data that isn’t accurate anyway?  Although it is true that accurate data is pretty useless if you can get access to it, the converse is also true.  What is the point of a great end-user enabled system that includes inaccurate data?

My Top 5 Customers – Really?

Take a look at the report below.  (If you want to download this Xcelsius Model it is available below.)

Who are my top 5 customers?

Top 10 Customers

Top 10 Customers

Did you say:  General Electric, Procter & Gamble, PepsiCo, Home Depot and Walmart?

Well, Sorry.  I’m afraid, that would be incorrect.

You see, what often happens in real-world situations is that organizations think they have more customers than they actually do.  That’s because within their CRM system, employees are able to add the same customer multiple times with multiple spellings.  This has happened in our case as well.  Let’s apply BusinessObjects Data Quality to this real-world situation.  With SAP BusinessObjects, you can take company names, customer names, addresses, etc. and standardize them, e.g. UPS = United Parcel Service = UPS Inc., WalMart = Wal*Mart = Wal-Mart, First Commerce Bank = 1st Commerce Bank.

My Top 5 Customers – Really!

Let’s have a look at this same report with Data Quality applied:

Top 10 Customers with Data Quality

Top 10 Customers with Data Quality

Do you see the changes?

Walmart has jumped up into second place and United Parcel Service is now in fifth place.  We can also see the our profitability at Walmart is higher than we thought (26.8% instead of 18.7%) and United Parcel Service is actually lower that we thought (28.6% instead of 26.3%).  When you are making business decisions off your corporate data, it’s imperative that it is accurate and complete.

Here is the source data behind this chart and you can see how the lack of standardization has led to the incorrect results.  I have highlighted the offending records for you:

Raw Customer Data

Raw Customer Data Behind the Top 10 Customers Report

Once we apply data quality and standardize the names, the order changes and I have a new top 5!  Often times our biggest customers, vendors, partners and products don’t get the credit they deserve for contributing to our success.  Once you’ve got data quality, you can know that you know that you know, the true numbers.

I’ve introduced this topic under the name of Data Quality, but Data Quality really falls under the broader topic of Data Stewardship or Data Governance.

You Don’t Know What You Don’t Know

The bottom line around data quality is that you don’t know what you don’t know.  If you manage a data warehouse which accepts feeds from dozens of systems, then it’s highly likely that you have a data quality problem and don’t even know it.  It’s a critical aspect of data warehousing.  Operational systems are notorious for bad data.  Last year, I read an excellent, practical guide to data quality called, Data Quality Assessment.  The book itself does not endorse a specific software vendor but all the principles found in the book would apply to any organization looking to improve their corporate data quality.

Downloads – See It Live

If you’d like to see an Xcelsius model of this chart live, I’ve made it available for download.  The source code for the .xlf is also available: of Data

Truth Is Stranger Than Fiction

Sometimes in life you run across situations that are hard to believe.  Here is an example where truth is stranger than fiction.  When you want to get someone’s attention when it comes to data quality, just tell them this example.  This data quality situation really happened and the results were disastrous.  This video is from Timo Elliott. When you click on it, it will take you to his website:

Data Quality Issues

Timo's Data Quality Presentation (2min)

Do you have any good stories to share?  I’d love to hear them.

«Good BI»

Teleran – Your Database Nanny!

March 19th, 2008 No comments

Teleran LogoOkay – so the auditing and query management that comes with SAP BusinessObjects doesn’t do it for you? Need more umph? Do you want more real-time information about your BusinessObjects system? Are you experiencing query and database performance problems? Need to identify long running queries and kill them?

Every once and a while you hear about a partner technology that makes you say, Wow! And that is what I said when I saw a demonstration of the solution from Teleran.

These guys are amazing and have a great solution for customers that are running Microsoft SQL Server, Oracle or DB2 (no Teradata yet). They have created specialized listeners in their iSight tool that watches the traffic and logs the activity in real-time. They even have a universe and reports already built using SAP BusinessObjects so that you can get up and going quickly. If you find a particular query is taking to long, perhaps you need to tune the universe and tweek the database. Teleran can show you the problem.

Teleran Manages Query Performance

If you are having problems with user generating bad SQL, out-of-compliance queries, or if you simply want to add additional governs around the system they also have a tool called iGuard. iGuard is integrated directly into tools like WebIntelligence, so instead of a user getting a message like, “Query Error ODBC – Unable to Connect”, then can get a message like, “The product server is currently down for maintenance. Please try again in 25 minutes”. Now THAT is user friendliness.

Because Teleran understands how SAP BusinessObjects works, even if you are using a single user id such as “report_user” at the database level, Teleran can read the user, universe and connection, etc. information and let you easily audit who is doing what at the database level. Don’t worry about turning on database logging and slowing down your production database. You can offload the work to Teleran.

The Greatest Part of All

The greatest part of Teleran is not that it’s a great solution for SAP BusinessObjects managers and administrators, it’s great for the entire EDW. Why? Because Teleran runs between the database and the query tool and is NOT SAP BusinessObjects dependent. You see, it can monitor ALL queries – not just queries from BusinessObjects. This means that if you have users access the database via TOAD, you can turn it off. You can turn off access to any unauthorized reporting/SQL generating tool.

To me, this seems like a must have solution for organizations who are struggling to get a handle on their data warehouse and need to make sure they are delivering a robust and high performing environment.

Check it out! You’ll be glad you did.