“All models sweep dirt under the rug. A good model makes the absence of the dirt visible.” Emanuel Derman
There’s been a lot of talk buried in the reaction from Friday’s job report and the disparity between the BLS’s report and the ADP report; some people say that it’s irrelevant and offer up a misguided list of reasons. Others seem to recognize that the ADP number is meant to be used as an input into the overall picture and not the end all to be all when it comes to “forecasting” the NFP number. Even in the latter case, some people give the ADP numbers too much weight in painting the overall picture. The ADP number is still particularly relevant but many people want to take the easy way out and either outright dismisses it as a flawed number, citing that sample size issues or taking a crack at the somewhat opaque nature of how they arrive at the numbers, etc. It shouldn’t be dismissed outright in my view, but over reliance on that number to make a decision or prediction about the NFP number is foolhardy.
Emanuel Derman wrote about the over-reliance on models many times and I tend to agree with him. If I had solely relied on my crude attempt at modeling the NFP number I would have been way off from reality. Joe Weisenthal made an interesting point discussing about the differences between seasonal and non-seasonal adjustments seen in June’s report and arrived at a number that was somewhat close to the crude “model” spit out for me in the early morning hours on Friday. I was tempted to change my “prediction” from +30k to a new number of +134k all based on what this model told me. But I didn’t.
Why? Economists and those stuck living in a pure mathematical/model world seldom venture and interact with the real world. If they actually did on the ground research, they would see that the trends are not overall positive for the labor market. If they found themselves amongst the unfortunate who are experiencing this brutal labor market firsthand (I hope they don’t, it’s not a pretty place to be) they would also surely take a different view. The point is: models do not capture fully what is occurring in real life. Dana Telsey’s approach worked for her – ground level research of visiting stores, talking with people, asking the right questions. Sure, she used models to assist her in making calls/predictions, but she did not come to solely rely on them to tell her what was going on in the real world.
The same thing can be said about the ADP, NFP and all these other economic numbers. Many can create a fancy mathematically elegant model but as Derman has discussed at length; one can become blinded and caught up in the beauty of the model that you fail to forget what is really going on in the underlying world that you are attempting to model. These numbers and statistics are meant to be used as an input into your decision making process about things. Just like many things in life, people assign various “weights” to inputs on decision making. Perhaps if you view the number as being so rife with errors, drop it entirely or assign a lower weight to it.
Models don’t capture everything, no matter how mathematically elegant or rigorous they are. They’re meant to be used as a tool. To solely rely on them to paint picture of what’s really going on in the real world is foolish. Get out and talk to the people that are being “modeled” as opposed to sitting isolated and saying that the problem isn’t as bad (or good) as people are making it out to be.