Posts Tagged ‘Segmentation’

Set Analysis Part 3b: Creating Sets In BusinessObjects

June 22nd, 2008 3 comments

w that we’ve spent some time with sets, it’s time to begin to see the technology in action. Business Objects has made sets easy to build and maintain. There is virtually no limit to the ways you can now choose to look at your customers and look at how those customers are contributing to revenue, profit and margins.

Using Sets to Maximize Revenue

Click HERE is a quick online demo that we’ve built to explain how some of our telecommunications customers use sets to maximize revenue, cross sell and up sell opportunities.

Creating Sets

The beauty of sets is the fact that you don’t have to involve IT in making changes to the underlying data structures to segmentation. With the set tables that come with Set Analysis, end users can create their own custom collections of products, customers, etc. BusinessObjects users will then be able to use those sets together with the BusinessObjects Suite to provide in depth reporting of key metrics and kpis with respect to these custom collections of customers. Whether you want to track standardized silver, gold and platinum customers or time, or simply do some adhoc customer analysis, sets gives you what you need.

In XI 3.0 you can create sets using our Performance Manager dashboard interface. A set can contain data for only one subject. The subject is normally identified as a primary key in a dimension table. In the case of the Customer subject, the Customer Id key will be the key we are tracking.

Create Calendar

The single remaining configuration component that still remains is to create a calendar. Calendars are used by BusinessObjects to allow administrators to automate task such as refreshing sets and to allow us to track the changes of sets over time. We need to tell BusinessObjects how often track the changes in our sets.

Based on continuing our demo from the my previous post, lets create a calendar monthly. To create calendar, navigate to the Dashboard & Analytics Setup. Next, choose the Time Config > Calendar submenu. Here is where you can see a list of all available calendars. Let’s choose New Calendar

Based on the same data provided in my previous post, we will create a calendar:

  1. Name: DM_Monthly
  2. Interval: Monthly
  3. Format: First Day of Period, 9/21/04
  4. Range: Jul-00 to Mar-03

After you have created the calendar the screen should look like this:

Create Calendar in BusinessObjects

NOTE: Once a calendar has been created, you cannot change values such as interval.

Now that the setup is complete, we can move to Dashboards & Analytics. Chose the Set Analysis > Sets submenu.

The first thing we need to do is create a folder so that we can categorize our sets. Choose New Folder and let’s name our folder: Customer Types.

Create New Folder

Let’s also create a second folder called: Customer Segments.

Now let’s create our first set.

Static Sets

The first set we will create will be a static set. A static set is a collection of objects that either:

  • Doesn’t change over time or
  • if it changes, we are not concerned about tracking those changes.

Some examples of static customer sets are:

  1. Customers Living in Montreal
  2. Homeowners with more than one child
  3. Key accounts younger than 30 years old
  4. Customers in the UK with sales over 10,000 in 2001

We will be able to create a set of customers using any of the customer table reference that we setup previously. Let’s create a set of all High Tech GenX Customers, that is: All Customers with Internet and in the 30-40, 40 – 50 Age Bands.

First we need to name our static set and set some attributes for our set. Our set will be called:

  1. High Tech GenX
  2. Folder: Customer Types
  3. Description: This set will contain a list of all customers who are considered Highly Technical, Generation Xers.
  4. Scope: Public
  5. Set Type: Static

Creating a Static Set in Set Analysis

Click Apply. Now we will specify the content of the set. In our case, we will use Visual Data.

Create Set Content

NOTE: The interface can be a little tricky. You will see Add/Copy/Remove buttons next to Sets and also find Add/Edit/Remove/Empty button within the Set Building Sets section. The buttons on the left operate on the entire set itself. The buttons on the right only affect the selected building step.

There are 5 was to add information into a set. They are:

  • Visual Data – Allows you to look at all the fields associated with your subject (e.g. customers) and visually see how many records will be included in the set. (Most Popular)
  • Visual Sets – Allow you to reuse existing sets to create new sets. For example you can visually see the intersection between High Tech GenX Customers and Gold Customers for example.
  • Free Hand SQL – Allows the users to specify their own SQL against the database. (Not Recommended)
  • Event Analysis – Allows you to create new sets based on a records movement within a dynamic set for a specific time period. For example, all customers who were leavers between Oct-02 and Dec-02.
  • Import – Allows you to manually import records (e.g. contact_id values) from a csv or txt file.

We will chose Add > Visual Data and be taken to the window, where we can select the attributes of our set members.

Creating a Static Set in Set Analysis - Step 3

We will chose:

  • Has Internet – Y
  • Age Range – 30-40 and 40-50

We will select the attribute on the left and choose the arrow to move the attribute to the right. (If prompted, chose Exact Counts). The system will tell you that 1135 contacts have Internet. Choose the Y checkbox.

Next select the Age Band attribute on the left and expand the section. You will see that of those with Internet, 321 are aged 40-50 and 302 are aged 30-40. Select both of these checkboxes. This means that when we build the set, we will expect a total of 623 members.

Now choose Next and fill in the Description: Internet and Aged 30-50

Now we can choose OK and we can see the resulting set.

Creating a Static Set in Set Analysis - Step 4

We could continue to add additional steps, which would allow us to modify this set further (allowing us to add or subtract members), but our example will only have one step. Now we need to build the set.

Building Sets

Building Sets is the process of the Set Analysis engine using the definition of the set to generate the appropriate SQL statement(s) that brings back the correct records. It will then insert those records into our set tables for future reference.

In the screenshot above, you can see the cursor next to the Build… icon. Click build now and watch as the set engine identifies 623 members for the set. Until you build the set, your set will have zero members. You an always see how many members a step and the last date/time the set was rebuilt.

If at any point you edit the building sets of a static set, the set will be automatically emptied (all set members will be deleted) and you will need to rebuild the set.

Dynamic Sets

The process of creating dynamic sets is very similar to that of static sets, except the set will be processed for specific time periods. Let’s create a dynamic set of Gold Customers. Gold Customers are customers who have made volume purchases and purchased more than $3000 of products from us in each period. We will again use the Visual Data to build our new set.

First we need to name our static set and set some attributes for our set. Our set will be called:

  1. 1 – Gold Customers
  2. Folder: Customer Segments
  3. Description: Our Worldwide Gold Customers
  4. Scope: Public
  5. Set Type:Dynamic
  6. Calendar: DM_Monthly
  7. Membership History: Temporal (allows entire history of set membership to be kept)
  8. Allow Auto-Fill: Yes

Creating a Dynamic Set in Set Analysis

Click Apply. Now we need to build our first step. Let’s choose Add > Visual Data.

First, when specifying dynamic, temporal sets, it is important to specify the time element as the first component of the set detail. We want to make sure we are only including customers who have made a purchase is the current period. Because the sales date is part of the Orders Subject, we need to choose DM Orders.

Creating a Dynamic Set in Set Analysis - Step 2

Use the arrow to move the Start Date to the right hand side. You will be prompted to specify the Added Field Condition. We will need to specify, Start Date, between, with the values of Relative, Fixed, start of processing period and Relative, Fixed, end of processing period. The Yes/No condition should be selected.

In our case we will see that 18107 records are No. By default No may be selected. Make sure and change it to Yes.

TIP: Usually you will want Yes/No conditions to be Yes, so under Options (above the first Yes/No Condition line), you can chose the setting: Default Yes/No conditions to Yes.

Next, let’s look for Contacts who have purchase at least 12 items from us in a single monthly period. So chose Sum of Volume. You need to be careful here. Notice how we chose Sum of Volume. This is critically important, otherwise it would only choose customers with sales of one transaction with a Volume of 12. That’s not what we want. We want to look at total volume for the entire period.

Finally we want Contacts who have purchased more than $3000 of items in the same period. Let’s chose Sum of Actual Sales greater than 3000.

Now chose Next>> and enter a description: High Volume, High $$ and click OK.

Creating a Dynamic Set in Set Analysis - Step 3

After creating the set, BusinessObjects gives you the option to Autofill the set. The Autofill process uses the set engine to populate the historical values for the given set to a predetermined number of periods. In our case, we will autofill all 17 monthly periods from Jun-00 to Sep-03.

Click on AutoFill/Rollback…, Complete (radio button) and OK.

Creating a Dynamic Set in Set Analysis - Step 4

Now the request to process the set historically will be sent to the Set Analysis engine and after it processes all 32 periods, you can click on the Membership view and see the an historical view of the set.

Creating a Dynamic Set in Set Analysis - Step 5

Here you can see each period and the number of joiners, members and leavers.

Create two more sets for Silver and Bronze customers with volume purchases. I recommend Silver as Volume: 12 or more, Sales: 1000 – 3000. I recommend Bronze as Volume: 12 or more, Sales: <1000.

NOTE: If you want to get more members for each period, you can drop the volume requirements and only look at Amount of Sales.

Tips For Organizing Sets

It is important to note that you cannot create subfolders, so I recommend you create development standards when it comes to sets. Because all folders and sets are organized in alphabetical order, I recommend that you use the colon “:” to differentiate folder depth.

For example:


In the same way, use colons, numbers and names to organize your sets (1, 2, 3 or A, B, C will help create order.) For example, if I am going to create three sets which are mutually exclusive, such as Gold, Silver, Bronze, I would name them:1-Gold, 2-Silver, 3-Bronze. Now they will appear in the correct order. If however I am going to put more than 1 set in the folder, I might called it ProjA:1 – Gold, ProjA:2 – Silver and ProjA:3 – Bronze. That way I can call the next ones ProjB, etc (or whatever project name made sense in my situation).

Try to keep set names to less than 25 characters.

Consider using other symbols such as asterisk(*), plus(+), etc at the end of the set name to indicate addition important information (such as the fact that this set depends on another set).

If you don’t organize your sets, they will become cluttered. Another advantage of these types of naming conventions is that all this set information is stored in a relational database, so we can create BusinessObjects reports which can help us keep our sets organized. We can parse out colons an other special characters which may have special meaning to the organization.

Read more…

Set Analysis Part 3a: Setting Up Sets Metadata

June 18th, 2008 No comments

I have found that setting up the Metadata around Sets was one of the most confusing things about learning sets. The problem was two-fold. The interface of Set Architect is old and not terribly intuitive. It is little more than a database table browser with a few rules built in.

I typically learn by example, so I put together what I felt was the best combination of screenshots to show you the core of what you will likely run into when you need to configure Set Architect.

To download the sample database that we used for this example, please click here.

You may find trouble reading through the Set Architect manual before install the Set Analyzer client because the document isn’t available on our website. The document is contained within the installer program but I thought it might be a good idea to make it available here in case you need it.

  • Set Analyzer Administrator Guide – Administration and Setup Guide, which primarily uses Set Architect (required for using Sets)
  • Set Analyzer User Guide – Legacy documentation on how to use the Set Analyzer client tool. This tool was made obsolete because the primary interface for sets has been integrated into the Business Objects Performance Manager dashboard interface.

Metadata Configuration within SetArchitect

There are four core components of the metadata that need to be configured from within Set Architect. They are the:

  • Tables/Objects – the database tables and the field definitions we will use when generating SQL to create our sets.
  • Subjects – traditionally the dimension tables. Subjects are types of data that we might want to create a set of. For example Customers, Products, Suppliers, Contacts
  • Lookups – tables that are used to lookup decodes, e.g. customer status, type, category.
  • Converters – advanced metadata that tells Set Analysis how to convert a query from one subject to another. Converters are used for the most powerful types of sets.

Let’s look first at the database tables and database fields metadata, which is called Tables and Objects respectively.

Configuring Tables/Objects Metadata

We need to define the metadata for four tables in our example: CONTACTS, SECTORS, PRODUCTS and FACT.


The first table we needed to set up is customer. The unique key for this table is Contact Id. Because the customer table is not a lookup table, we indicate that this is indeed a source table.

Most of this metadata can be imported automatically when you use the Load Tables and Load Fields button at the top of the interface.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

EDW Contacts

I made only a few modifications to the default metadata that was imported using the import tables button. I added:

  • to contact_id, the linked table DM Contacts
  • to sector_id, the linked table DM Contact Sector
  • a new field called Annual Income Band with the following formula (SQL Server Syntax):
    case when <a>.annual_income = 0 then ‘Unknown’ when <a>.annual_income < 30000 then ‘<30K’ when <a>.annual_income < 50000 then ’30-50K’ when <a>.annual_income < 75000 then ’50K-75K’ when <a>.annual_income < 125000 then ’75K-125K’ else ‘125K+’ end
  • a new field called Age Band with the following formula (SQL Server Syntax):
    case when <a>.age < 30 then ‘<30′ when <a>.age < 40 then ’30-40′ when <a>.age < 50 then ’40-50′ else ’50+’ end

The new fields could have been defined within a database view (which is what I would recommend), but I wanted to demonstrate how they can also be added directly from within the metadata of Set Analysis. The syntax will be database specific. (There is much more information about this in the Set Architect Manual)


The second table we needed to set up is sector. The unique key for this table is Sector Id. Because the sector table is a lookup table for customer, we indicate that this is indeed a Lookup Object and NOT a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

EDW Sectors

I made only one modifications to the default metadata that was imported using the import tables button. I added:

  • to sector_id, the linked table DM Contact Sectors

As I’ve mentioned before you can define lookup tables from within Set Architect, but why? Why not create a new view and join the Contacts and Sectors tables into a common view on the sector_id? By pushing this to the database, you can simplify your configuration and you don’t have to worry about a lookup tables with lookups (e.g. state -> country -> continent).


Next is the product table. The unique key for this table is Product Id. Because the product table is not a lookup table, we indicate that this is indeed a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

EDW Products

I made only one modifications to the default metadata that was imported using the import tables button. I added:

  • to product_id, the linked table DM Products


The fourth and final table we needed to set up is fact table. There is no unique key for this fact table, but that doesn’t matter because we won’t be creating sets on top of orders. (If we needed to, then we would need an order_key which could be inserted into the set_ tables for linking). Because this fact table is not a lookup table, we indicate that this is a source table.

Here is a screenshot of the completed definition. (Click on the image to enlarge)

EDW Fact Orders

I made only a few modifications to the default metadata that was imported using the import tables button. I added:

  • to contact_id, I added the linked table DM Contacts
  • to product_id, I added the linked table DM Products
  • a new field called Quarter with the formula (SQL Server Syntax):
    case when <a>.week between 1 and 13 then ‘Q1’ when <a>.week between 14 and 26 then ‘Q2’ when
    <a>.week between 27and 39 then ‘Q3’ else ‘Q4’ end

Keep in simple. In real life, if you wanted to add additional calculations like quarter or profit (sales_actual – cost), then do it in the database or through a database view. This will make your life easier when it comes to long term maintenance.

Once all the tables have been defined, we can move to Subjects.


Subjects of a set are the central theme of the set. it will be associated with the table that most uniquely describes the subject for which sets are created. In our example, we are using contacts and products. We want to be able to create sets of both contacts and sets of products, so we will define two subjects.

Contact Subject

Here is how we will define the Contact Subject. (to enlarge click on picture)

Define Contacts

Product Subject

Here is how we will define the Product Subject. (to enlarge click on picture)

Define Contacts

Next, we need to check our lookups. This information should already be defined based on the changes we made when defining the sector table and defining it as a lookup for contacts.


Here is how we will define the metadata for the sector lookup. (to enlarge click on picture)

Define Lookups


Converters are just plain cool. When I went through training on sets, I never really grasp what it is they do or their role in defining sets. Converters are used to allow you to convert from one subject to another. Clear as mud? Well, actually they allow you to define sets based on the “related” subject. Here are a couple of great examples:

  • The set of customers that buy these products
  • The set of products that have been purchased by these customers
  • The set of other products sold with these products

So now you can track the set of Gold Customers who have purchased Premium Products during the period.

Converting Contacts to Products

The SQL we used for our converter was:
select distinct PRODUCT_ID as ID
where CONTACT_ID in ( <sub_query> )

Define Converter

Converting Products to Customers

The SQL we used for our converter was:
select distinct CONTACT_ID as ID
where PRODUCT_ID in ( <sub_query> )

Define Converter

The help text is completely optional and is not exposed to Set Analysis within the Performance Manager dashboard.


We’ve had an opportunity to walk through an example that covers the most common scenarios that I’ve seen in the field. If you have additional questions or comments, please list them below. I will use the database and metadata associated with this article to demonstrate the power of sets.

Read more…

Set Analysis Part 2: The Basics

June 16th, 2008 No comments

So How Exactly Does Set Analysis Work?

Set Analysis works together with the tables that are already part of your Data Warehouse. For example, in your Data Warehouse today you have customers, products and orders, etc. Therefore all the tables that below to Set Analysis must reside in the same database as your datasource. (From this perspective, I like was SQL Server allows you to do. In SQL Server, as long as you specify the database.owner.table, you can have the database tables in physically different database instances. It works because they can share the same connection.)

Installing SetAnalyzer

The first thing you might have problems with is finding the Set Analyzer software. Although the engines for using Set Analyzer are built into the BusinessObjects Platform, you still need to install the SetArchitect legacy client-server software to do some minimal configuration.

Traditionally BusinessObjects provided SetArchitect as a seperate download; however starting in XI 3.0, it is located with your BusinessObjects XI 3.0 installation media under the Add-Ons directory. The actual file path is: D:BOEXI30_Install_Fileswin32_x86CollateralsAdd-OnsSetArchitect.

You may have trouble finding the documentation for the legacy Set Architect client application. The documentation will be installed when you install the Set Architect Add-on from the BusinessObjects XI 3.0 installation media. However if you want to see the documentation prior to the installation you can download it here:

NOTE: Traditionally Set Analyzer existed as a client-server application, but once it was integrated into the BusinessObjects Platform, it no longer became necessary to use Set Analyzer. Business Objects has moved all but some very basic configuration steps to the web-based Performance Manager interface.

Set Tables

The tables which belong to the Set Analysis application all belong with set_. There are just over 100 tables in all. These tables definitions get created one of two ways. The set_ tables are created when:

  1. The Business Objects administrator creates the repository from the Dashboard and Analytics Setup Screen. From within System SetupRepository, the administrator defines the repository connection and “Creates” the Performance Manager tables. When using this method, not only are the set tables created, but the tables for all the BusinessObjects Performance Manager Engines are created. (Standard Method)
  2. The Set Analysis Administrator uses Set Analyzer Architect to “Setup” the repository. This method will only create the set tables for the Set Analyzer application. (Legacy Method – Not Recommended)

Once the Set Analysis tables have been created, you will be able to begin configuring your Set Analysis Data Sources. In a typical BusinessObjects Performance Manager Repository, this is what you would see:

Performance Manager Repository

Defining Your Data Sources

When setting up Set Analysis for the first time, there are only two tasks that need to be performed from within the legacy Set Architect application.

  1. You will need to define a connection, so that Set Architect understands how to connect to the database which contains your data warehouse data and the set analyzer tables.
  2. Next, you will need to set the metadata for your data warehouse tables so that set analyzer understands how to query the underlying database and create the sets.

Optimizing Your Data Sources and Set Analyzer

I highly recommend that if possible, you create database views on top of all the data sources that you plan to use for creating sets. For example if you have a customer table with 5 or 10 associated look-up tables for things such as geography, demographics, customer details, etc; then join these tables together and create a master view. Be aware that it is very important that your view NOT contain duplicate customer keys.

in Set Analyzer you need to define subjects. These are non-lookup tables associated with the fact table. In traditional data warehousing venacular they would be called dimension tables. These dimension tables need to be defined as Subjects in Set Architect.

If there is a many-to-many relationship between two tables, then I would need to create two subjects for each of the two tables. For example, if products have many suppliers and suppliers supply multiple products, (e.g. a many to many relationship exists), then I will need to define suppliers and products as Subjects. This will allow me to create complex sets such as:

  • All suppliers who supply less than 10 products.
  • All products which are supplied by suppliers in Georgia, who supply at least 5 products.

In my next post, I’ll provide screenshots of a sample metadata configuration as well as an excel spreadsheet for you to download and use yourself. All you will need to do is import the data into your favorite database.

Set Analysis – The Good Old Days

June 13th, 2008 3 comments

Old Leather JacketDo you have an old leather jacket? You know the kind. The jacket that reminds you of the good old days. Each time to try it on, it fits so perfectly, you wonder why you don’t get it out more often. It’s tough, it’s cool, it’s unique, and everyone should have one. When I bring it out and show the kids – they love it.

Well, from a software perspective, my “old leather jacket” is Set Analysis. I recently had a situation where a customer needed the capabilities of Set Analysis in their organization to solve some specific reporting needs. I had to dust off the manuals and fire it back up again, but when I did, I wondered why we’d been apart so long.

Set Analysis is a product that was created by Next Action and was acquired by Business Objects way back in October of 1999. Business Objects then launched a company called Ithena, which created analytic application and use set analysis extensively. I provide this background to let you know that the Set Analysis product has been around for a long time and it still brings value today.

Set Analysis provides ability to categorize information. The fancy term for this is called segmentation and is typically market or customers segmentation (although it also can relate to products, employee, etc.) This capability is critical to help organizations understand how the customers and products are affecting their business. We use collections, or sets, to make lists of items to communicate and to manage.

In this Venn diagram you can see there are three sets:

  • Last year’s gold customers
  • Current silver customers
  • Customers at risk

Segmentation products let Set Analysis make it easy to pick where you want to focus. For example the light green area represents the collection of: Last year’s gold customers who are at risk but are not currently silver.

Here is another example related to mortgage providers:

  1. Customers who have a good credit rating
  2. Customers who are paying an above average interest rate
  3. Customers who have not refinanced in the last 24 months.
  4. Customers who have never made a late payment.

Customers who exist in a set are called members.

Next, a mortgage company might take all four collections and market those customers who are members of all four sets with the goal of offering them a more competitive interest rate before they shop their mortgage with a competitor.

So what’s the big deal?

I hear what you are saying. What is the big deal? That’s just a normal query and you are right it is. What is facinating is the ability within a sets engine like Set Analysis to see the trends in these sets over time. Although it’s possible to get a snapshot of these collection members through a standard database query, what you don’t see is changes to the sets over time. For example you need to know:

  1. How many Platinum Customers do I have? 1,000.
  2. If I had 900 last month, did I have 100 join or did I have 200 new customers join and 100 leave?
  3. Where did they go? Did they become Gold? Silver? Are they off the radar?

It’s fascinating to me to see how organizations are so focused on what they sell and yet there is so much value in looking at what you didn’t sell… and who you didn’t sell it to!

In a down economy like we see today, it seems even more important than ever before to understand what your customers are doing.

In the next few posts, I’ll take a little bit of a deeper dive into the product and share a few tricks and tips.