Data Miner Survey - results
Some time ago I suggested that readers might like to complete a survey for Karel Rexer of Rexer Analytics and a number of you did. The results of the survey are now available here (you need to email Karl and ask for a copy, instructions on the page), and there are some useful nuggets of information in the report. A couple of things struck me when I saw my copy:
- When asked to identify the fields in which they employ data mining the most frequently identified fields were CRM/marketing, financial, academic, telecommunications, and retail.
Interestingly all categories on the blog with the exception of academia.
- Data miners working with financial data strongly valued a tool's ability to automate repetitive tasks
I wonder if this relates to the tendency of financial institutions to constantly update their models and develop multiple alternatives for use in adaptive control
- There was nothing on deployment
I found this curious as deployment of models seems really important to me. Not clear if Karl did not ask any questions or did not find the answers interesting (Karl?)
- Top 4 challenges found by respondents included dirty data, difficult access to data, explaining data mining to others, and finding qualified data miners
No big surprises here, apart from the explaining one - while I always find that a problem I was surprised to see it come up so high
- Other problems included that data mining results were not used by business decision makers and difficulties in deployment/scoring
I think using business rules as a platform for deploying analytics can help with both of these - it makes actual deployment easier and allows you to engage business people in the automation of the decision (through editing rules) in a way that might make them feel more comfortable.
Karl references some polls on KDNuggets about which I blogged before - one on what people do with analytics (here, CRM and banking came top), one on data mining deployment (here) and this one about making better decisions with computers (here).