Glance AI Series is a set of interviews where we invite smart accomplished individuals from digital marketing, data science, and AI to chat about interesting things. In this article Data Scientist Parth Agrawal explores the challenges of AI and Machine Learning and how the Pandemic has exposed some shortcomings of the same.
The last pandemic, the Spanish Flu, occurred in 1918. Back then, we didn’t have an understanding of data-driven decision making let alone a computer to crunch numbers. It is obvious that we have limited information on past pandemics, even as we were caught up in a new one century later. The more we try to dig into the past for answers, the more futile it seems. Somewhere, someone must have been prepared for this, right? Some models eating up unnecessary computer resources on a researcher’s machine must have predicted the outcome of this situation. Did someone’s predictive model allow them to make a killing during this pandemic while most of us are struggling to adapt to the new normal after so many months?
In this rat race of creating the perfect model, do you think there weren’t any hiccups caused by Covid-19? Let’s figure it out.
Act1: Data is the new oil and we are drowning in it
“Garbage in, garbage out.” This has been the mantra circling in the data science community since serious computing began. Deep learning and neural networks courtesy of high powered GPUs and data explosion has only made it worse. During this pandemic, we have not been at our very best. Our behaviors have changed (toilet paper hoarding, the decline in civility), our spending has become erratic, our minds are on the edge, and so is our data.
Let’s take an example of a simple recommender system that tells you to buy certain product X based on your purchase and browsing history. Here’s a quick look at how it looked before and how it looks now.
Unfiltered and messy data coupled with a huge causality factor called coronavirus has inadvertently skewed every model created – from the banking sector to marketing, every model mimicking human behavior has been pushed to its extreme limits where they don’t make sense at all.
The result: Shelving of many R&D projects and people reverting back to the tried and tested human intelligence. Will this push the envelope in the wrong direction? Probably. Will those who actually understand and adapt survive? Definitely. Is it good to be a Data Enthusiast now? Probably not.
Act 2: AI will kill traditional jobs but it won’t kill the virus
Enthusiasm for AI agents and RPA is high these days. The reason is obvious: Less workforce, more automation. Hiring computers, firing humans. But does AI as a replacement work in a situation like we have in 2020?
It has been more than six months now since COVID-19 burst on the scene. All the computing power in the world, all the biggest scientific models of the world, and they still haven’t been able to create the perfect vaccine to cure the virus. One would argue that many of them are in clinical trials, but do you think we can’t simulate the human body in this age where we are already close to simulating human behavior? There are still boundaries that no one can cross and they will probably remain for a long, long time. Forcing something unnatural is only going to create an imbalance in the force causing these disturbances from time to time.
The result: We are still learning to live with the virus. The second wave (and winter) is already here. We expect our models to adapt, vaccines to work, machines to make every mundane job to go away. But the fact remains that we need people on the frontlines. We need human decision making at the edge and not AI in pandemics because of the two things AI systems lack – accountability and empathy.
Act 3: The age of resistance
As we adapt to the pandemic, our models are learning to adapt too. Millions of patterns and countless strategies are shifting; some are working well and driving decisions.
The mystery and Black Box nature of these models do not bring about the desired confidence in them. For example, if you read our report on COVID-19 analysis, you will see that after countless analyses, with the help of domain expertise, we were able to draw conclusions and provide some strategies to navigate your digital marketing spends and efforts in such times.
However, if you let a generic tool like AutoML run haywire on our data, you will end up with some crazy strategies which are probably not there in any playbooks on the planet. This is the result of combining large amounts of data with skewed analysis driven by erratic behaviors which are the result of COVID. Do you trust the strategies a generic tool produces if they don’t make logical sense to you? As a veteran operator in marketing, probably not.
But if you are a desperate marketer who wants to save their job you might try them. If you don’t do something soon you are getting fired so might as well play a king’s gambit. And if you do, let me remind you of the age-old computing mantra: garbage in, garbage out. It doesn’t only apply to the data, but to the processes, we enact upon it as well.
At Glance, they are dedicated to unearthing valid patterns and give scientific reasons to justify the selection of such strategies. There is a hybrid approach with rigorous thresholding based on statistical analysis and then let ML algorithms take over. This gives control and justification for the recommendation suggested. Does this strategy make sense? Well, we can dive into that in the next article. Till then, think about the ways to blend traditional statistical analysis with new-age ML and DL models and you might just have a “pied piper” moment.
Epilogue – where the magic happens
It’s important to solve for the problems that matter, on hand, and are practical. Not all AI is meaningful and machine learning won’t solve all our problems. However, with a certain focus like Glance, who is solving the problems of various marketing data silos, metrics, channels, the lack of interconnectedness in the data, and generating smart insights, you can generate meaningful results e.g. effective marketing experience. So, choose your models well and keep in mind that a model will never be the all-seeing eye and silver bullet without your inputs.
Quick Bio: Parth Agrawal is a data scientist with four years of experience building data-driven products and services in Logistics, Edtech, Marketing, Smart City Solutions, and Agritech. He was the Initial data scientist who helped Glance with early models for marketing analysis. Currently, he dreams of electric sheep at night while building cutting-edge learning resources for teaching AI to the masses with Andrew Ng’s team @deeplearning.ai while loves to experiment on his mini garden with statistics helping him grow the tiniest vegetables you will ever see (or it is just dumb luck coupled with sheer ignorance)
Featured image credit: The Scientist magazine