David Taylors Business Intelligence Blog

Remember the Mouse revolution? I’ll never forget the first time I saw a Macintosh and used the mouse to move a pointer across the screen. I could start programs, paint pictures and do all kinds of cool things without learning a new interface.

Well, Jeff Han recently introduced the world to an entirely new generation of user interface during the annual TED conference. It is a Minority Report moment.

Its a new multi-sensor touchscreen which allows the user interface to completely disappear. You have to see it to believe it. It is truly incredible.

http://www.ted.com/index.php/talks/jeff_han_demos_his_breakthrough_touchscreen.html

So how we do do this exactly? How can I use BusinessObjects to create custom groups on-the-fly? How can I get beyond my requirements for multi-pass SQL and speed up my reports? The answer is Sets.

Set Data

All the information regarding sets is located is the tables called sets_*. The most important tables are:

• set_set – contains the set name, description, etc.
• set_set_detail – contains all the set member information including member key, join date, leave date.

Here is a universe I created that you can download and use with the sample database provided in my earlier blog post. We need to add the set_set and set_set_detail tables to our WebIntelligence Universe so that the set member information they contain can be used within our query.

Here you can see that we created a join to the DW_CRM_CONTACT table. We also created a self-join on the set_set_detail table. The self join make sure that we reference the edate value of 12/31/2999, which is the default edate for current members of any set. In the case of a dynamic set, If a member had left the set then the edate would not be equal to 12/31/2999.

In addition to this, we also joined the set_set table to the set_set_detail table using the setversion_id key. (It may appear that the setversion_id is always the same as the set_id, but it is not so only join the tables via the setversion_id)

Here is a look at the Custom Product Groups

BEST PRACTICE: Whenever you may need to alias a table. If you alias a table, then all references to that table should be aliased. You can see that because I needed to reference the set_set_detail and set_set tables twice, once for products and once for contacts, I aliased both.

User Prompts

After joining the tables, we needed to create the filters for our custom groupings to make it easy for users to select which groups they would like to use on the report.

I specifically defined the filters a multi, which will allow the user to select more than one filter. This is important because it provides even more power for your end-users. Now they can do direct comparisons of one group to another, even if a member (e.g. contact) exists in more than one group. The only downside is that it means that a member can be counted twice, this is why I recommended that when you define your sets you should use naming conventions that identify the sets as being mutually exclusive.

Here is what the prompt will look like in action:

In my universe I did not use cascading prompts to first limit the sets by folder, but that is certainly an option since folder information is located in the set_project table.

Getting Results

After configurating the universe and setting up the prompts this is an example of the results you will see. I have run a report and selected multiple custom groups which are mutually exclusive. These groups on based on the current periods sales amounts:

Here you can see the report is broken down by my custom groups which were defined and processed by the Sets Engine.

Summary

I’ve really enjoyed taking you on a quick tour of Set Analysis and showing you some of the ways that you might get more value out of your existing data assets. Sets offers so many solutions around data analysis:

• Custom Groupings – which allows the organization to more nimble as changes such as mgmt structures, mergers and acquisitions take place.
• Classic Segmentation – Joiners, Leavers, Stayers (which in turn are perfect for analysis by a predictive tool so that you can see why are people joining or leaving? What are the influencers?)
• Advanced Analysis – “Give me all quad-pack customers who have never purchased phone accessories from a company owned retail store.”

The ability to divide customers up into categories of: highly profitable down through :unprofitable, and using that to drive the business because the bottom line of any business is the customer and your organization’s ability to make a profit.

Although dashboards are currently in vogue and everyone wants one, I often wonder how organizations are analyzing the information beyond the dashboards. I have nothing against dashboards, but I would encourage you to ask yourself this. Once we see that sales are falling, what am we going to do? How am we going to do root cause analysis? Were all customers affected? What about products? Segmentation might well hold the answer to these yet unsolved mysteries.

I recently had a customer who was in the process of migrating from a legacy version of Microstrategy to BusinessObjects. They had been able to migrate over most of their core reports, but there were several reports that required a number of custom groups.

Microstrategy’s approach was to allow the customer to define any number of groups and filters from with the Microstrategy Desktop tool. These global filters and group definitions can then be leveraged by the report editor for building the report. Microstrategy’s SQL generation engine uses multi-pass SQL when creating the reports. Multi-pass SQL means that a single SQL statement cannot deliver back the correct results, so the report data is written into temporary tables. These temporary tables are then combined with additional queries to obtain the final results.

In the case of BusinessObjects, it wasn’t going to be possible to take the same approach. BusinessObjects does not generate multi-pass SQL. Instead, BusinessObjects focuses on creating extremely complex SQL in a single SQL statement. In most cases, this works just as well, but this was an exception. This sales report used a combination of multiple filters at multiple levels in the heirarchy. The Heirarchy was also defined within the custom groupings and not within the database.

The right answer is: Modify the Data Warehouse and ETL processes to make the underlying data better fit your reporting requirements. You see, the problem was that these custom groups should be defined within the data warehouse natively. This would allow for simplified SQL that can be run in a single pass. Translation? Reports the use custom groups will run a heck of a lot faster.

In my situation, the customer wasn’t going to be able to make these changes, but still wanted to migrate the report to BusinessObjects.

The best answer is: Use Sets. Sets gives you the ability to generate those custom grouping within the data warehouse. Unlike Microstrategy which will run the same multi-pass SQL over and over again, sets will allow us to “cache” the previous passes using sets and run single-pass SQL. We can then tie the set tables into the data warehouse and add the custom groupings to our universe.

The advantage of sets over ETL is that it can be much more dynamic. Sets can allow an analyst to create a new set in a matter of minutes and then see a report broken down by the new custom group.

In my case it was just what the doctor ordered. By introducing sets and adding the set tables to the BusinessObjects semantic layer, we now have access to the sets, which in this case, the customer called custom groups. It was perfect.

Now, not only will the customer be able to migrated this complex report from Microstrategy to BusinessObjects, but they will also be able to run the report much faster than ever before because the report will be generated through a single-pass process.

Tomorrow, we’ll take a look at how to set this up using our simplified data.

Since the merger with SAP, we’ve begun to adopt most of the SAP policies. One policy is related to vacation. SAP strongly encourages employees to take their vacation each year and not allow it to pile up. In fact, the maximum amount of vacation you are allowed to roll over each year in 5 days. This has lead some individuals to get serious when it comes to using up their vacation.

Here is an email reply I received from a colleague recently:

Now that’s what I call a vacation!!

A less common way to create sets is to create sets is through free-hand SQL. Normally you will want to leverage to power of the set engine to design the SQL for you to return the correct results, but in rare cases you might find it helpful to create your own SQL Statements.

Free Hand SQL

The only item that needs to exist in the SELECT clause is ID. If an ID is specified and the SQL is valid, then the records will be imported into the set tables, if not, you will not be able to chose “Parse” to validate your SQL.

For our example, we can create a set called “All Customers” and use the following FreeHand SQL.

select CONTACT_ID as ID from dbo.DW_CRM_CONTACT

The SQL may be as complex as you require. As long as the SQL is valid, you should be able to insert it here and use it with Set Analysis.

I am not aware of a limit for the Free Hand SQL.

Import

Set Analysis also provides the ability to import individual records into the set tables. This is valuable if users have exported information into Excel and manually developed a list of records they wish to add to a set.

You can either store the text file in the BusinessObjects Repository or on the physical filesystem. Unfortunately the schema definition is pretty limited, so I recommend you use one of the sample files below as a guide. As long as the file contains at least 1 column and the key is the FIRST column, the import will work fine.

Click on the Import button and pick your file. Do not type in the name manually. If necessary, you may need to define a file format. I recommend that you use comma delimited files with a double-quote text qualifier (”).

Here is a sample import file of DM Contacts.

NOTE: In order to use the Upload feature, a c:\tmp directory must be defined on your system.

NOTE: If you store the file on a filesystem, you may need to make sure that the Sets Query Service (AAQueryMgr.exe) has rights to access the file across the network. By default all BusinessObjects services run as LocalSystem which means by default they will not have access to UNC file paths
.

I’ve decided it’s probably a good idea to break down these tutorials a bit. Although I love to share all this information, it’s probably getting a little long for a single post.

Visual Sets

When creating sets in Set Analysis, Visual Sets are extemely helpful. Visual Sets allow you to quick look at the union, intersection, exclusion of multiple sets simultaneously. Let’s say for example I want to see all Gold Customers who are also High Tech Customers but have NOT purchased products from our computer department. Let’s assume I already have the following three sets:

• Gold Customers
• High Tech Customers
• All Products from the Computer Department

Let’s create a new static Set using DM Contacts, called Gold Customer – No Computer. Next we will add a Visual Set. We will chose 3 sets. One set for each of the three Collections.

• Collection 1(blue) – add Gold Customers (All Gold Customers)
• Collection 2 (red) – add High Tech (All High Tech Customers)
• Collection 3 (yellow) – add Computer (All Products in the Computer Department)

In our case we want all Gold Customers who have NOT purchase Products in the Computer department, therefore we do not want to select anything in the yellow circle.

I can click on Counts at any time and the system will show many how many records are in the selected set.

From the selected venn diagram regions, I can see from the displayed count that there are 78 customers who fall into this category. Click Next>>, specify the description and OK to save the set.

Remember, just like before we must build the set before it will contain any objects. Once you build the set you will see there are 78 records retrieved.

Isn’t that easy? How cool is that?

NOTE: Remember how we specified Converters in Set Analyzer Architect? This converter allows us to chose information from DM Products, but have it converted products to contacts. In our example, it converted “products in the computer department”, to contacts who have purchased products from the computer department.

w that we’ve spent some time with sets, it’s time to begin to see the technology in action. Business Objects has made sets easy to build and maintain. There is virtually no limit to the ways you can now choose to look at your customers and look at how those customers are contributing to revenue, profit and margins.

Using Sets to Maximize Revenue

Click HERE is a quick online demo that we’ve built to explain how some of our telecommunications customers use sets to maximize revenue, cross sell and up sell opportunities.

Creating Sets

The beauty of sets is the fact that you don’t have to involve IT in making changes to the underlying data structures to segmentation. With the set tables that come with Set Analysis, end users can create their own custom collections of products, customers, etc. BusinessObjects users will then be able to use those sets together with the BusinessObjects Suite to provide in depth reporting of key metrics and kpis with respect to these custom collections of customers. Whether you want to track standardized silver, gold and platinum customers or time, or simply do some adhoc customer analysis, sets gives you what you need.

In XI 3.0 you can create sets using our Performance Manager dashboard interface. A set can contain data for only one subject. The subject is normally identified as a primary key in a dimension table. In the case of the Customer subject, the Customer Id key will be the key we are tracking.

Create Calendar

The single remaining configuration component that still remains is to create a calendar. Calendars are used by BusinessObjects to allow administrators to automate task such as refreshing sets and to allow us to track the changes of sets over time. We need to tell BusinessObjects how often track the changes in our sets.

Based on continuing our demo from the my previous post, lets create a calendar monthly. To create calendar, navigate to the Dashboard & Analytics Setup. Next, choose the Time Config > Calendar submenu. Here is where you can see a list of all available calendars. Let’s choose New Calendar

Based on the same data provided in my previous post, we will create a calendar:

1. Name: DM_Monthly
2. Interval: Monthly
3. Format: First Day of Period, 9/21/04
4. Range: Jul-00 to Mar-03

After you have created the calendar the screen should look like this:

NOTE: Once a calendar has been created, you cannot change values such as interval.

Now that the setup is complete, we can move to Dashboards & Analytics. Chose the Set Analysis > Sets submenu.

The first thing we need to do is create a folder so that we can categorize our sets. Choose New Folder and let’s name our folder: Customer Types.

Let’s also create a second folder called: Customer Segments.

Now let’s create our first set.

Static Sets

The first set we will create will be a static set. A static set is a collection of objects that either:

• Doesn’t change over time or
• if it changes, we are not concerned about tracking those changes.

Some examples of static customer sets are:

1. Customers Living in Montreal
2. Homeowners with more than one child
3. Key accounts younger than 30 years old
4. Customers in the UK with sales over 10,000 in 2001

We will be able to create a set of customers using any of the customer table reference that we setup previously. Let’s create a set of all High Tech GenX Customers, that is: All Customers with Internet and in the 30-40, 40 – 50 Age Bands.

First we need to name our static set and set some attributes for our set. Our set will be called:

1. High Tech GenX
2. Folder: Customer Types
3. Description: This set will contain a list of all customers who are considered Highly Technical, Generation Xers.
4. Scope: Public
5. Set Type: Static

Click Apply. Now we will specify the content of the set. In our case, we will use Visual Data.

NOTE: The interface can be a little tricky. You will see Add/Copy/Remove buttons next to Sets and also find Add/Edit/Remove/Empty button within the Set Building Sets section. The buttons on the left operate on the entire set itself. The buttons on the right only affect the selected building step.

There are 5 was to add information into a set. They are:

• Visual Data – Allows you to look at all the fields associated with your subject (e.g. customers) and visually see how many records will be included in the set. (Most Popular)
• Visual Sets – Allow you to reuse existing sets to create new sets. For example you can visually see the intersection between High Tech GenX Customers and Gold Customers for example.
• Free Hand SQL – Allows the users to specify their own SQL against the database. (Not Recommended)
• Event Analysis – Allows you to create new sets based on a records movement within a dynamic set for a specific time period. For example, all customers who were leavers between Oct-02 and Dec-02.
• Import – Allows you to manually import records (e.g. contact_id values) from a csv or txt file.

We will chose Add > Visual Data and be taken to the window, where we can select the attributes of our set members.

We will chose:

• Has Internet – Y
• Age Range – 30-40 and 40-50

We will select the attribute on the left and choose the arrow to move the attribute to the right. (If prompted, chose Exact Counts). The system will tell you that 1135 contacts have Internet. Choose the Y checkbox.

Next select the Age Band attribute on the left and expand the section. You will see that of those with Internet, 321 are aged 40-50 and 302 are aged 30-40. Select both of these checkboxes. This means that when we build the set, we will expect a total of 623 members.

Now choose Next and fill in the Description: Internet and Aged 30-50

Now we can choose OK and we can see the resulting set.

We could continue to add additional steps, which would allow us to modify this set further (allowing us to add or subtract members), but our example will only have one step. Now we need to build the set.

Building Sets

Building Sets is the process of the Set Analysis engine using the definition of the set to generate the appropriate SQL statement(s) that brings back the correct records. It will then insert those records into our set tables for future reference.

In the screenshot above, you can see the cursor next to the Build… icon. Click build now and watch as the set engine identifies 623 members for the set. Until you build the set, your set will have zero members. You an always see how many members a step and the last date/time the set was rebuilt.

If at any point you edit the building sets of a static set, the set will be automatically emptied (all set members will be deleted) and you will need to rebuild the set.

Dynamic Sets

The process of creating dynamic sets is very similar to that of static sets, except the set will be processed for specific time periods. Let’s create a dynamic set of Gold Customers. Gold Customers are customers who have made volume purchases and purchased more than $3000 of products from us in each period. We will again use the Visual Data to build our new set.

First we need to name our static set and set some attributes for our set. Our set will be called:

1. 1 – Gold Customers
2. Folder: Customer Segments
3. Description: Our Worldwide Gold Customers
4. Scope: Public
5. Set Type:Dynamic
6. Calendar: DM_Monthly
7. Membership History: Temporal (allows entire history of set membership to be kept)
8. Allow Auto-Fill: Yes

Click Apply. Now we need to build our first step. Let’s choose Add > Visual Data.

First, when specifying dynamic, temporal sets, it is important to specify the time element as the first component of the set detail. We want to make sure we are only including customers who have made a purchase is the current period. Because the sales date is part of the Orders Subject, we need to choose DM Orders.

Use the arrow to move the Start Date to the right hand side. You will be prompted to specify the Added Field Condition. We will need to specify, Start Date, between, with the values of Relative, Fixed, start of processing period and Relative, Fixed, end of processing period. The Yes/No condition should be selected.

In our case we will see that 18107 records are No. By default No may be selected. Make sure and change it to Yes.

TIP: Usually you will want Yes/No conditions to be Yes, so under Options (above the first Yes/No Condition line), you can chose the setting: Default Yes/No conditions to Yes.

Next, let’s look for Contacts who have purchase at least 12 items from us in a single monthly period. So chose Sum of Volume. You need to be careful here. Notice how we chose Sum of Volume. This is critically important, otherwise it would only choose customers with sales of one transaction with a Volume of 12. That’s not what we want. We want to look at total volume for the entire period.

Finally we want Contacts who have purchased more than $3000 of items in the same period. Let’s chose Sum of Actual Sales greater than 3000.

Now chose Next>> and enter a description: High Volume, High $$ and click OK.

After creating the set, BusinessObjects gives you the option to Autofill the set. The Autofill process uses the set engine to populate the historical values for the given set to a predetermined number of periods. In our case, we will autofill all 17 monthly periods from Jun-00 to Sep-03.

Click on AutoFill/Rollback…, Complete (radio button) and OK.

Now the request to process the set historically will be sent to the Set Analysis engine and after it processes all 32 periods, you can click on the Membership view and see the an historical view of the set.

Here you can see each period and the number of joiners, members and leavers.

Create two more sets for Silver and Bronze customers with volume purchases. I recommend Silver as Volume: 12 or more, Sales: 1000 – 3000. I recommend Bronze as Volume: 12 or more, Sales: <1000.

NOTE: If you want to get more members for each period, you can drop the volume requirements and only look at Amount of Sales.

Tips For Organizing Sets

It is important to note that you cannot create subfolders, so I recommend you create development standards when it comes to sets. Because all folders and sets are organized in alphabetical order, I recommend that you use the colon “:” to differentiate folder depth.

For example:

Level1
Level1:Level1a
Level2
Level2:Level2a
Level2:Level2b

In the same way, use colons, numbers and names to organize your sets (1, 2, 3 or A, B, C will help create order.) For example, if I am going to create three sets which are mutually exclusive, such as Gold, Silver, Bronze, I would name them:1-Gold, 2-Silver, 3-Bronze. Now they will appear in the correct order. If however I am going to put more than 1 set in the folder, I might called it ProjA:1 – Gold, ProjA:2 – Silver and ProjA:3 – Bronze. That way I can call the next ones ProjB, etc (or whatever project name made sense in my situation).

Try to keep set names to less than 25 characters.

Consider using other symbols such as asterisk(*), plus(+), etc at the end of the set name to indicate addition important information (such as the fact that this set depends on another set).

If you don’t organize your sets, they will become cluttered. Another advantage of these types of naming conventions is that all this set information is stored in a relational database, so we can create BusinessObjects reports which can help us keep our sets organized. We can parse out colons an other special characters which may have special meaning to the organization.

Read more…

I have found that setting up the Metadata around Sets was one of the most confusing things about learning sets. The problem was two-fold. The interface of Set Architect is old and not terribly intuitive. It is little more than a database table browser with a few rules built in.

I typically learn by example, so I put together what I felt was the best combination of screenshots to show you the core of what you will likely run into when you need to configure Set Architect.

To download the sample database that we used for this example, please click here.

You may find trouble reading through the Set Architect manual before install the Set Analyzer client because the document isn’t available on our website. The document is contained within the installer program but I thought it might be a good idea to make it available here in case you need it.

• Set Analyzer Administrator Guide – Administration and Setup Guide, which primarily uses Set Architect (required for using Sets)
• Set Analyzer User Guide – Legacy documentation on how to use the Set Analyzer client tool. This tool was made obsolete because the primary interface for sets has been integrated into the Business Objects Performance Manager dashboard interface.

Metadata Configuration within SetArchitect

There are four core components of the metadata that need to be configured from within Set Architect. They are the:

• Tables/Objects – the database tables and the field definitions we will use when generating SQL to create our sets.
• Subjects – traditionally the dimension tables. Subjects are types of data that we might want to create a set of. For example Customers, Products, Suppliers, Contacts
• Lookups – tables that are used to lookup decodes, e.g. customer status, type, category.
• Converters – advanced metadata that tells Set Analysis how to convert a query from one subject to another. Converters are used for the most powerful types of sets.

Let’s look first at the database tables and database fields metadata, which is called Tables and Objects respectively.

Configuring Tables/Objects Metadata

We need to define the metadata for four tables in our example: CONTACTS, SECTORS, PRODUCTS and FACT.

CONTACT TABLE AND FIELDS

The first table we needed to set up is customer. The unique key for this table is Contact Id. Because the customer table is not a lookup table, we indicate that this is indeed a source table.

Most of this metadata can be imported automatically when you use the Load Tables and Load Fields button at the top of the interface.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only a few modifications to the default metadata that was imported using the import tables button. I added:

• to contact_id, the linked table DM Contacts
• to sector_id, the linked table DM Contact Sector
• a new field called Annual Income Band with the following formula (SQL Server Syntax):
  case when <a>.annual_income = 0 then ‘Unknown’ when <a>.annual_income < 30000 then ‘<30K’ when <a>.annual_income < 50000 then ‘30-50K’ when <a>.annual_income < 75000 then ‘50K-75K’ when <a>.annual_income < 125000 then ‘75K-125K’ else ‘125K+’ end
• a new field called Age Band with the following formula (SQL Server Syntax):
  case when <a>.age < 30 then ‘<30′ when <a>.age < 40 then ‘30-40′ when <a>.age < 50 then ‘40-50′ else ‘50+’ end

The new fields could have been defined within a database view (which is what I would recommend), but I wanted to demonstrate how they can also be added directly from within the metadata of Set Analysis. The syntax will be database specific. (There is much more information about this in the Set Architect Manual)

SECTOR TABLE AND FIELDS METADATA

The second table we needed to set up is sector. The unique key for this table is Sector Id. Because the sector table is a lookup table for customer, we indicate that this is indeed a Lookup Object and NOT a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only one modifications to the default metadata that was imported using the import tables button. I added:

• to sector_id, the linked table DM Contact Sectors

As I’ve mentioned before you can define lookup tables from within Set Architect, but why? Why not create a new view and join the Contacts and Sectors tables into a common view on the sector_id? By pushing this to the database, you can simplify your configuration and you don’t have to worry about a lookup tables with lookups (e.g. state -> country -> continent).

PRODUCT METADATA

Next is the product table. The unique key for this table is Product Id. Because the product table is not a lookup table, we indicate that this is indeed a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only one modifications to the default metadata that was imported using the import tables button. I added:

• to product_id, the linked table DM Products

FACT METADATA

The fourth and final table we needed to set up is fact table. There is no unique key for this fact table, but that doesn’t matter because we won’t be creating sets on top of orders. (If we needed to, then we would need an order_key which could be inserted into the set_ tables for linking). Because this fact table is not a lookup table, we indicate that this is a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

I made only a few modifications to the default metadata that was imported using the import tables button. I added:

• to contact_id, I added the linked table DM Contacts
• to product_id, I added the linked table DM Products
• a new field called Quarter with the formula (SQL Server Syntax):
  case when <a>.week between 1 and 13 then ‘Q1′ when <a>.week between 14 and 26 then ‘Q2′ when
  <a>.week between 27and 39 then ‘Q3′ else ‘Q4′ end

Keep in simple. In real life, if you wanted to add additional calculations like quarter or profit (sales_actual – cost), then do it in the database or through a database view. This will make your life easier when it comes to long term maintenance.

Once all the tables have been defined, we can move to Subjects.

DEFINING SUBJECTS

Subjects of a set are the central theme of the set. it will be associated with the table that most uniquely describes the subject for which sets are created. In our example, we are using contacts and products. We want to be able to create sets of both contacts and sets of products, so we will define two subjects.

Contact Subject

Here is how we will define the Contact Subject. (to enlarge click on picture)

Product Subject

Here is how we will define the Product Subject. (to enlarge click on picture)

Next, we need to check our lookups. This information should already be defined based on the changes we made when defining the sector table and defining it as a lookup for contacts.

DEFINING LOOKUPS

Here is how we will define the metadata for the sector lookup. (to enlarge click on picture)

DEFINING CONVERTERS

Converters are just plain cool. When I went through training on sets, I never really grasp what it is they do or their role in defining sets. Converters are used to allow you to convert from one subject to another. Clear as mud? Well, actually they allow you to define sets based on the “related” subject. Here are a couple of great examples:

• The set of customers that buy these products
• The set of products that have been purchased by these customers
• The set of other products sold with these products

So now you can track the set of Gold Customers who have purchased Premium Products during the period.

Converting Contacts to Products

The SQL we used for our converter was:
select distinct PRODUCT_ID as ID
from DW_CRM_FACT_ORDERS
where CONTACT_ID in ( <sub_query> )

Converting Products to Customers

The SQL we used for our converter was:
select distinct CONTACT_ID as ID
from DW_CRM_FACT_ORDERS
where PRODUCT_ID in ( <sub_query> )

The help text is completely optional and is not exposed to Set Analysis within the Performance Manager dashboard.

CONCLUSION

We’ve had an opportunity to walk through an example that covers the most common scenarios that I’ve seen in the field. If you have additional questions or comments, please list them below. I will use the database and metadata associated with this article to demonstrate the power of sets.

Read more…

When I was growing up I always enjoyed putting puzzles together. I loved the challenge of putting all those pieces together so that I could see the whole picture. Sometimes I would borrow a puzzle from a friend and as I was close to finishing the puzzle, I would discover 1 or 2 puzzle pieces were missing. The puzzle was incomplete.

From a BI perspective, APOS has been a long partner of BusinessObjects. My first opportunity to work with APOS came while I was working at Crystal Decisions. Our next generation Crystal Enterprise product was new and although it provided a great framework for the future, there were a few pieces missing from our complete BI picture. Thankfully, APOS developed a number of powerful solutions to fill those gaps so that our customers didn’t have to wait until tomorrow’s release for the functionality they needed today.

Do you feel you are missing a piece from BusinessObjects XI 3.0?

Business Objects has many great partners, but probably non-better than APOS. In January of this year, APOS was awarded the Business Objects Technology Partner of the Year Award for 2007. They have used the BusinessObjects SDK more extensively than any other BusinessObjects partner. They don’t just one solution. Their tools provide dozens of potential missing pieces that might fit your BI puzzle.

My Favorite APOS Tools

So what’s so great about APOS? Well, here are my top 3 list favorite solutions:

1. InfoScheduler – mass scheduling – dead simple. Scheduling has always been one of those areas were BusinessObjects added a few new features with ever release. Unfortunately however it’s not an area that tends to get a lot of attention. It was only in XI 3.0 that we introduced native bursting within the platform. InfoScheduler provides an Excel-based report scheduling tool that allows mass scheduling of reports from an Excel spreadsheet. How easy is that? Can you say zero training? Users can copy paste, import, export data into Excel to provide scheduling for 10, 100 or 1,000s of reports.
2. Object Manager – Although the CMC is a comprehensive administration console, administrators of large reporting environment often struggle with managing this volume of information. In XI 3.0, BusinessObjects introduced a number of new enhancements which made working with multiple objects much easier; however Object Manager takes it to the next level. You can manage users, groups, reports, security and even the report life cycle process. Object Manager provides power, ease-of-use and flexibility.
3. KPI – One of the most common requests I hear from customers is the ability to do reporting off their CMS. BusinessObjects XI provides an auditing database to allow customers to report off of user activity, but how do administrators report off the objects within the CMS? With KPI, APOS allows you to extract all the XML metadata that is stored within the CMS to an external relational database. Now you can create reports that allow you to:

• Analyze Business View and Universe configuration information
• Examine Crystal and Desktop Intelligence report structure
• See How many reports use the database field profit
• Show CMS Statistics
• Discover how many reports are scheduled to run tomorrow.

You can even set an automatic schedule to refresh the external metadata database on a regular basis. It’s your own mini-datamart for reporting off your BI system. You have to see it to believe it.

Southwest is using all three of these tools and several more!

Can’t Find Your Missing Piece?

If you’ve look through the APOS website and you can’t find the type of functionality that you feel is missing from BusinessObjects, call APOS anyway. Necessity is the mother of invention. Each of the solutions that APOS provides today was developed from the real-world needs of customers just like you. APOS would love to talk to you about extending BusinessObjects in a way that they may not have thought of.

Puzzle Complete

As a kid I was rarely able to find the missing puzzle piece, but in the area of extending BusinessObjects, APOS continues to deliver. Their depth of experience means they understand how BusinessObjects works today and how to make it work better for you tomorrow.

So How Exactly Does Set Analysis Work?

Set Analysis works together with the tables that are already part of your Data Warehouse. For example, in your Data Warehouse today you have customers, products and orders, etc. Therefore all the tables that below to Set Analysis must reside in the same database as your datasource. (From this perspective, I like was SQL Server allows you to do. In SQL Server, as long as you specify the database.owner.table, you can have the database tables in physically different database instances. It works because they can share the same connection.)

Installing SetAnalyzer

The first thing you might have problems with is finding the Set Analyzer software. Although the engines for using Set Analyzer are built into the BusinessObjects Platform, you still need to install the SetArchitect legacy client-server software to do some minimal configuration.

Traditionally BusinessObjects provided SetArchitect as a seperate download; however starting in XI 3.0, it is located with your BusinessObjects XI 3.0 installation media under the Add-Ons directory. The actual file path is: D:\BOEXI30_Install_Files\win32_x86\Collaterals\Add-Ons\SetArchitect.

You may have trouble finding the documentation for the legacy Set Architect client application. The documentation will be installed when you install the Set Architect Add-on from the BusinessObjects XI 3.0 installation media. However if you want to see the documentation prior to the installation you can download it here:

• Set Architect Documentation
• Set Analyzer Documentation

NOTE: Traditionally Set Analyzer existed as a client-server application, but once it was integrated into the BusinessObjects Platform, it no longer became necessary to use Set Analyzer. Business Objects has moved all but some very basic configuration steps to the web-based Performance Manager interface.

Set Tables

The tables which belong to the Set Analysis application all belong with set_. There are just over 100 tables in all. These tables definitions get created one of two ways. The set_ tables are created when:

1. The Business Objects administrator creates the repository from the Dashboard and Analytics Setup Screen. From within System Setup\Repository, the administrator defines the repository connection and “Creates” the Performance Manager tables. When using this method, not only are the set tables created, but the tables for all the BusinessObjects Performance Manager Engines are created. (Standard Method)
   
2. The Set Analysis Administrator uses Set Analyzer Architect to “Setup” the repository. This method will only create the set tables for the Set Analyzer application. (Legacy Method – Not Recommended)

Once the Set Analysis tables have been created, you will be able to begin configuring your Set Analysis Data Sources. In a typical BusinessObjects Performance Manager Repository, this is what you would see:

Defining Your Data Sources

When setting up Set Analysis for the first time, there are only two tasks that need to be performed from within the legacy Set Architect application.

1. You will need to define a connection, so that Set Architect understands how to connect to the database which contains your data warehouse data and the set analyzer tables.
2. Next, you will need to set the metadata for your data warehouse tables so that set analyzer understands how to query the underlying database and create the sets.

Optimizing Your Data Sources and Set Analyzer

I highly recommend that if possible, you create database views on top of all the data sources that you plan to use for creating sets. For example if you have a customer table with 5 or 10 associated look-up tables for things such as geography, demographics, customer details, etc; then join these tables together and create a master view. Be aware that it is very important that your view NOT contain duplicate customer keys.

in Set Analyzer you need to define subjects. These are non-lookup tables associated with the fact table. In traditional data warehousing venacular they would be called dimension tables. These dimension tables need to be defined as Subjects in Set Architect.

If there is a many-to-many relationship between two tables, then I would need to create two subjects for each of the two tables. For example, if products have many suppliers and suppliers supply multiple products, (e.g. a many to many relationship exists), then I will need to define suppliers and products as Subjects. This will allow me to create complex sets such as:

• All suppliers who supply less than 10 products.
• All products which are supplied by suppliers in Georgia, who supply at least 5 products.

In my next post, I’ll provide screenshots of a sample metadata configuration as well as an excel spreadsheet for you to download and use yourself. All you will need to do is import the data into your favorite database.