Clik here to view.
Why Predictive Modelers Should be Suspicious of Statistical Tests (or why the...
Well, the danger is really not the statistical test per se, it the interpretation of the statistical test. Yesterday I tweeted (@deanabb) this fun factoid: "Redskins predict Romney wins POTUS #overfit....
View Article6 Reasons You Hired the Wrong Data Miner
As is in any discipline, talent within data mining community varies greatly. Generally, business people and others who hire and manage technical specialists like data miners are not themselves...
View ArticleTop Posts in 2012
For the second consecutive year, a quick look back at posts from the prior year.For posted in 2012, in order of popularity:Target, Pregnancy, and Predictive Analytics, Part ITarget, Pregnancy, and...
View ArticleThree Ways to Get Your Predictive Models Deployed
We all know that given reasonable data, a good predictive modeler can build a model that works well and helps make makes better decisions than what is currently used in your organization (at least in...
View ArticleWhen Analysis Isn't the Answer
Data mining is an important tool whose benefits have been demonstrated in diverse fields, among business, government and non-profit organizations. Its application areas continue to grow, especially...
View ArticleUsing Geographic Data
Most organizations collect and maintain some type of geographic data, yet many ignore this data during analysis. Any business has some record of customer addresses, for instance, but this data is...
View ArticleWhat To Take Home from Your Next Predictive Analytics Conference
Why should one go to a predictive analytics conference? What should one take home from a conference like Predictive Analytics World (PAW)? There are many reasons conferences are valuable including...
View ArticleClik here to view.
Do Predictive Modelers Need to Know Math?
(Note: this post was first published in the March 2013 Edition of the Predictive Analytics Times) Predictive analytics is just a bunch of math, isn’t it? After all, algorithms in the form of matrix...
View ArticleMath and Predictive Analytics - A Personal Account
Last week I taught a workshop at Predictive Analytics World entitled Supercharging Prediction: Hands-On with Ensemble Models. The workshop was intended to introduce predictive modelers to the concept...
View ArticleDean Abbott Featured in "Popular Mechanics" On-Line Article
Our own Dean Abbott has been consulted for an on-line Popular Mechanics article, "Why the NSA Wants All That Verizon Metadata" (Jun-06-2013), by Glenn Derene. Since the initial report connecting the...
View ArticleBig Data is Not Enough
Big data is the big buzz word in the world of analytics today. According to google trends, shown in the figure, searches for "big data" have been growing exponentially since 2010 though perhaps is...
View ArticleThe NSA, Link Analysis and Fraud Detection
The recent leaks about the NSA’s use data mining and predictive analytics has certainly raised awareness of our field and has resulted in hours of discussions with family, relatives, friends and...
View ArticleBeware Phantom Data
One of the perennial challenges facing the data analyst is missing values. A great deal has been written about the importance of identifying the source of missing values, the danger of overly...
View ArticleOn Data Mining Contests
Data mining contests have grown in popularity over the years, from the annual competitions at technical conferences to the continuous stream of events at sites like Kaggle. This has yielded several...
View ArticleA Good Business Objective Beats a Good Algorithm
Predictive Modeling competitions, once the arena for a few data mining conferences, has now become big business. Kaggle (kaggle.com) is perhaps the most well-known forum for modeling competitions,...
View ArticleSpeaking Engagements First Quarter 2014
I'll be speaking at several events this quarter1) EITA Global Webinar: Key Steps in Starting Your First Predictive Analytics ProjectTuesday, January 14, 2014, 1:00 PM ESTThis 90 minute webinar will...
View ArticleData Science and Big Data Search Trends
These are from google trends.Data science is growing, but still way behind other traditional terms for our field such as data mining, predictive analytics and machine learning. Big data on the other...
View ArticleClik here to view.
Why Overfitting is More Dangerous than Just Poor Accuracy, Part I
Arguably, the most important safeguard in building predictive models is complexity regularization to avoid overfitting the data. When models are overfit, their accuracy is lower on new data that wasn’t...
View ArticleWhy Overfitting is More Dangerous than Just Poor Accuracy, Part II
In Part I, I explained one problem with overfitting the data: estimates of the target variable in regions without any training data can be unstable, whether those regions require the model to...
View ArticleClik here to view.
Similarities and Differences Between Predictive Analytics and Business...
I’ve been reminded recently of the overlap between business intelligence and predictive analytics. Of course any reader of this blog (or at least the title of the blog) knows I live in the world of...
View ArticleData Mining's Forgotten Step-Children
Depending on whose definition one reads, the list of activities which comprise data mining will vary, but the first two items are always the same...Number 1: PredictionThe most common data mining...
View ArticlePredictive Modeling Skills: Expect to be Surprised
Excerpted from Chapter 1 of my book Applied Predictive Analytics, Wiley 2014Conventional wisdom says that predictive modelers need to have an academic background in statistics, mathematics, computer...
View ArticleTracking Model Performance Over Time
ContextMost introductory data mining texts include substantial coverage of model testing. Various methods of assessing true model performance (holdout testing, k-fold cross validation, etc.) are...
View ArticleA Question of Resource Allocation
Of the resources consumed in data mining projects, the most precious (read: "expensive") is time, especially the time of the human analyst. Hence, a significant question for the analyst is how best to...
View ArticleClik here to view.
Article 0
What Programming do Predictive Modelers Need to Know?Dean AbbottSmarterHQ and Abbott Analytics(First published in Predictive Analytics Times,...
View Article