A business intelligence system looks very complicated (see below). It's filled with legacy data, processes to extract and massage data, a new data warehouse to hold consistent and integrated data, and, for decision makers, a variety of user interfaces including summary and exception reports, analytic tools, ad hoc query, and graphical dashboards.
Building this interconnected system requires the following steps, not necessarily in this order:
1. Understand decisions, then design the user interface
2. Get support from the top and key user groups
3. Know where valuable data are
4. Clean and integrate data
5. Do not rely exclusively on project management from the BI vendor
6. BI for operational managers is easier than BI for senior management
7. Iteratively develop the Big Three: data warehouse, development tools, and user interface
Case Study: Big Bank Dashboard
A large multinational bank had five call centers spread across three time zones for their credit card operations, employing thousands of call center reps. Some of the call centers handled inbound calls from card holders with billing questions, others handled outbound calls to overdue accounts, while a third handled calls from merchants asking about payment dates and quantities. With this variety of call centers, there was a corresponding variety of applications available to the call center reps, five in all.
Management had a service level agreement (SLA) with the IT department at the bank. Although SLAs are typically created with external service providers, this was a smart move for the business. The SLA covered system availability, transaction response time, mean time between failures (MTBF), mean time to repair (MTTR), and several other factors. These technical metrics helped the call center operations deliver quality service to card holders and merchants – minimal wait times and short handle times.
These are simple, clear measures of quality service. Unfortunately, the underlying systems were not so simple. The five desktop systems available to the call center reps were supported by 16 backend systems and a myriad of interconnected networks and voice systems. If any one of them had trouble, operations would suffer.
Managers and IT staff had no real-time system for understanding or diagnosing problems. Quite the opposite, they only had a PowerPoint stack with dozens of static graphs showing system performance. The report came out about three or four weeks after the end of each month. These delays in delivering information made timely repair of problems impossible, although they were useful for post-hoc analysis and root cause repair.
We were engaged to design a dashboard for their business intelligence system after a usability review of the current system revealed its inadequacies. Naturally we pursued a user-centered design methodology: Stakeholder interviews, Manager interviews, Prototyping the dashboard, Usability test, and Refine and deliver.
What Information do Decision Makers Need?
"What gets measured, gets managed." – Peter Drucker
In order to understand what decision makers need, a good analyst should begin with a framework for understanding corporate objectives. The Balanced Scorecard, by Kaplan and Norton (1992), provides four dimensions to corporate objectives:
• Customer satisfaction
• Internal business processes
• The organization's ability to learn and improve
This framework is often used to identify key performance indicators (KPIs) which are simply performance measures, items that should be measured and improved to evaluate the success of an activity, department, or division. KPIs are just what we need to know if we are to build a BI system for decision makers.
Step 1: Stakeholder Interviews.
With this framework in mind, the business analyst can interview corporate stakeholders to understand the business objectives and needs for the BI system. This will typically produce no more than 20 measures spread across financial and non-financial topics.
Step 2: Manager Interviews.
Next, we can interview the decision makers and managers who will use the BI system. We have found that a combination of interviews and contextual inquiry provide thorough coverage of needs and wants. In our experience, the mid-level managers had several graphs, charts and reports that they relied on for post-hoc analysis. These charts and reports became parts of the BI's user interface, the dashboard.
Step 3: Prototyping the Dashboard.
Managers can understand data faster if it is presented well. In our experience, the bank managers needed a way to visualize the performance of their call center, so we prototyped a clickable, web-based dashboard. In the same way that a car's dashboard informs the driver about the car's performance, this digital dashboard showed them how their credit card operation was performing. It also let them drill down into details and to slice and dice the data with dynamic filters. The default configuration worked for most, although users could also customize it to create a personalized home screen.
Step 4: Test its Usability.
Usability testing will provide feedback on the prototype dashboard. In our situation, we created realistic scenarios involving typical problems. We gave each scenario to a manager, one at a time, and asked them to solve the problem using the prototype. We used a performance-based test to verify that managers could navigate quickly, and that they could access the KPIs for their operation.
Step 5: Iteratively Refine.
With feedback from usability testing, we refined the design with attention to the corporate color palette and appropriate logos.
The dashboard we created had several key features:
• customized home page with at a glance charts most relevant to each manager
• rapid navigation, three or four levels deep, to different reports and graphs, designed to match the users' mental models for the information architecture
• ability to change monthly/yearly time frame
• dynamic filtering of metrics by location, application and backend systems
Designing a great user interface for a business intelligence system will naturally flow from following a solid methodology, as we describe above. We also learned how important it is to spend time with managers, observing what they do. And when you find pain points, when you see Excel being stretched to its limits or when you see lots of data extracts and manual massaging by data analysts, you've found a candidate for expanding BI into a new realm.
It is key to create a convincing story. Build a vision of where to go. Then you want to tell the story to others to build support. Because of the power of pictures and images, carry a camera. Also, turn developers loose to build a working prototype quickly. Managers love seeing and using a working prototype to help them clarify what they need. And because time is important, leverage existing resources with outside help as needed to build a team that collaborates well. This will bring quick progress and a successful system that meets the needs of users and decision makers.
Information Management (2012). Gartner Reports Billions More for BI, Information Management, http://www.information-management.com/news/SAP-Oracle-IBM-Gartner-BI-analytics-PM-10022274-1.html
Kaplan, R.S. and Norton, D.P. (1992) The Balanced Scorecard – Measures That Drive Performance, Harvard Business Review, January-February 1992, pp. 71-79.