There’s an old joke about machine learning that goes: “If you walked into a job and just guessed a bunch of answers until you were right, you would get fired. But if you program a machine to do it, you get paid 6 figures as a ‘Machine Learning Analyst'”. The more I’ve tinkered with TensorFlow and Keras, the truer that joke has become.
My initial introduction to Machine Learning through TensorFlow and Keras quickly taught me three key things about Machine Learning. First, it’s mostly a test of how well you remember linear regression from your Statistics courses at University. Second, Machine learning programmers are guessing almost as much as their models most of the time, and third (which will be covered next week), machines are stupidly smart.
I found that the real key to being a master of Machine Learning lies not in your skill as a programmer (although it helps with the raw I/O handling of massive data dumps), but in your skill as a statistician and mathematician. The programming involved in actually making and training a model in Keras is relatively simple for any Python developer, but the real complexity and knowledge comes from your mathematical skill and knowledge to know why you should be adding a RELU activation layer, or for that matter what RELU even is, and why you’re using it. It’s fairly simple to follow and copy other’s machine learning code, but to actually understand the reasoning and functioning behind the processes is much more complex.
Of the next point, the amount of ‘playing by feel’ in machine learning was surprising. In trying to find an answer or a rule of thumb for the density and depth of layers to use when constructing a model, I found answers by PH.Ds in Machine Learning, individuals who are at the top of their scientific field answering the question with ‘try it with some random settings. If it doesn’t work, add more layers or remove layers, or add more channels and keep trying until it works better’. I guess much like how people say people’s dogs often look like their owners, it makes sense that machine learning scientists use so much guessing and adjusting based on results as their machine learning models use. I guess coming from a Neuroscience background where one research methodology for coronal function mapping in animals (in layman’s terms, what the different regions of the brain do) is to literally damage different parts of the brain with a scalpel, and observing what the animal can/can’t do to establish what that part of the brain does, but I was still surprised by the amount of guessing involved in machine learning.
This is all I have for this week, be sure to come back next week for the thrilling part 2 (and conclusion) to my thoughts and muses on machine learning.