Kirk Borne on Building Data Science Models

Tip 1: Think big, but start small. Someone once said, "Never use a big word when a small one will do." The same can be said of data science activities: "Never use a complex model when a simpler one will suffice." Similarly, Albert Einstein said that all things should be as simple as possible, but no simpler. The KISS principle applies: Keep It Simple and Smart.

The Pareto principle also applies: 80 percent of value comes from your first 20 percent of effort. You can read more about the 80-20 rule on my blog post here.

Tip 2: Don’t build the model, find it. Data science today should be focused less on "building the model" and more on "finding the model in the data.” In the big data era, the data becomes the model. What does this mean?

As a data scientist, your job is to decode the data, to discover the patterns (information, knowledge, and insights) that are already encoded in the bits and the bytes of the data collections that you have assembled. 

Example: Think about DNA sequencing. A DNA sequence is a symbolic representation of the sequence of about 20,000 genes. That symbolic representation (the bits and the bytes—i.e., the data) is essentially indecipherable. You can't make any sense of it, but nevertheless all the things you need to know about your heredity and possible future (propensity for diseases, hair loss, hair color, etc.) are encoded there. The knowledge is already there. The "model" to predict your health future is already there!

Therefore, the job of data scientists (in the DNA case, the job of bioinformatics  experts) is to decode the data and to find the knowledge in the data—i.e., to find the model in the data, because the data is the model.

Tip 3: Fail fast, learn fast. Data science is a fail-fast, learn-fast endeavor. To find a successful model, a data scientist must make at least two errors before declaring success.

Mathematically, this is called “gradient descent.” Here’s how it works.

First, you build one model and measure its accuracy (or error rate). That tells you nothing until you build a second model (refined, and hopefully improved, following the first model). You can then measure this second model's accuracy (or error rate). That now tells you whether your refinements in going from the first model to the second model were good (i.e., the model improved) or were bad (i.e., the model got worse).

From that information (two "failed" models), you learn which direction your model refinements should take to further reduce the error (i.e., keep going in the “good” direction, or turn around and stop going in the “bad” direction).

This process continues until you find the optimum model, where the accuracy (or error rate) reaches the “best” value: any further refinement of the model makes it worse, making the accuracy lower or the error rate higher.

This iterative, continuous-learning data science process is not sustainable nor affordable unless it is fast! Therefore, in data science, we aim to fail fast in order to learn fast.