individual-entry-BA-Blog

« Adaptive Control - Experimental Design | Main | Continuous Optimization »

Adaptive Control: Decision Analysis

Continuing (and completing) the series on Adaptive Control:

Decision analysis is the formal comparison of different approaches - Champion or Challenger - to see which ones work best given the organizations current strategy. It might also involve analysis to assess which will be more successful in the future. The result of this analysis might be to promote a Challenger, remove a Challenger and design a new one or to change the underlying premise for the experimental design. Ensuring that business users, especially those with responsibility for maintaining the rules in a deployed Decision Service, have the right environment to monitor the data will help them keep the decision running optimally. These users should have performance management and reporting tools that use the data in the operational systems to track how well the overall processes and systems are behaving. This kind of monitoring will help them see when they might need to make a change to a threshold in a rule or perhaps add or remove rules. The results feed into the formal analysis of Champions and Challengers.

Comparison can be of actual results to predicted results or of the results of two alternatives. Such comparisons can be at a specific moment in time or over a more extended period. Statistical analysis and standard reporting tools are all used to understand how well the deployed rules and models are working, how Champions and Challengers compare and to formulate strategies for improving things. Profit, retention and other classic measures can be tracked and compared for each challenger and the champion. Remember that you must be able to tell which approach was used for each customer/account/transaction as otherwise no comparison is possible.

One of the most effective tools other than standard reporting is a swapset analysis, like that illustrated below. This compares how different approaches or strategies apply various actions to different segments of your customer base (a classic thing to vary between champions and challengers).

SwapSet

You can also use standard graphs to compare the distributions and results across segments for different approaches. If you have built a decision model and are able to develop optimal solutions based on those models then you can also report your actual results against what is called an efficient frontier. This plots the best result possible for given levels of constraint - in the example below the profit per account given a change in losses. If your results map to the efficient frontier then you are doing as well as your constraints allow. Otherwise not.

EfficientFrontierGraphic

Finally you might want to read the section on Decision Yield as it contains some ideas for tracking decision yield over time and the various champion and challenger decisions could be tracked using those.

, , , , , , , ,

First time on the EDM blog?
Subscribe to the EDM blog feed or check out some other recent posts:

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451629b69e200d83465fb8069e2

Listed below are links to weblogs that reference Adaptive Control: Decision Analysis:

» Anothr feed track -Enterprise Decision Management - a Weblog from anothr user
One new subscriber from Anothr Alerts [Read More]

Comments

Post a comment

Comments are moderated, and will not appear on this weblog until the author has approved them.

If you have a TypeKey or TypePad account, please Sign In.

Search Site


  • dmblog.fico.com

Subscribe

  • enter your email

Upcoming Events

  • FICO Tools & Analytics User Forum 2012
    BERLIN: September 11-12, 2012 LONDON: September 18-19, 2012 Gain new insights for improving business performance through advanced analytics and decision management tools.