Fundamentals of AI, ML and Deep Learning for Product Managers
As promised in the previous post, we are going to describe in detail the powerful AI/ Machine-learning algorithms driving innovation at companies such as Quora, Uber, Netflix, Amazon among others.
Before that, we need to learn some basics.
This post covers those fundamental building blocks of AI.
What is AI?
Simply put AI is used to describe machines that mimic human-level intelligence while performing cognitive functions such as learning and Problem solving.
The term is further categorised into two: “Strong AI”- the pop-culture description of AI where self-sustaining machines can perform (and even outperform) humans in cognitive functions in varied and novel tasks. This is the long-term goal, computer scientists want to achieve, which has kept visionaries like Elon Musk worried for the future.
This is to differentiate from “Narrow AI,’ which refers to systems designed for one specific task. All current AI tools are Narrow AI. These systems such as Alpha Go (which beat the Chinese Go master in 2017) can outperform humans, but on very narrow tasks, such as the complex game Go here.
Also,Most AI systems currently available are essentially prediction tools. They help predict outcome given a certain input. The AI model takes in data and uses it to generate relevant information of past, present and future. For example: Classifying credit card transaction as fraudulent, identifying abnormalities on X-rays and MRIs or recommending what you would buy or watch or listen to in future.
Finally, AI is considered valuable because they can often produce better, faster, and cheaper predictions than humans can. Infact, now it can create its own AI software, more efficiently than human engineer can.
And How it works? Welcome Machine Learning
Most of the today’s powerful AI systems are powered by Machine Learning algorithms. ML is the field derived from Data Science and Computer Science which enables computers to learn and perform tasks without being explicitly programmed for every decision-rule.
Simply put, Every ML model utilizes three types of data:
(1) training data for training the AI
(2) input data which performs real-time predictions
(3) feedback data for improving the prediction accuracy of AI.
For example, abnormality detection medical imaging AI is first trained on well-classified images of lungs categorised as having/not-having lung cancer. Once the machine learns the difference between healthy lungs and cancer-ridden lungs, the system then is put into production to predict whether an unclassified lung has cancer or not. This information is then verified by the doctor and AI then gets back feedback whether it performed the prediction accurately or not. If not, it tweaks its own set of classification rules.
Trending AI Articles:
1. Machine Learning Concepts Every Data Scientist Should Know
2. AI for CFD: byteLAKE’s approach (part3)
3. AI Fail: To Popularize and Scale Chatbots, We Need Better Data
The different faces of AI
Once you start reading on AI/ML, you will get confused by varied terms: Natural Language Processing (NLP), Neural Networks, Deep Learning (DL) and so on. While the demarcations is still not very clear, all these are a sub-field of Machine Learning.
But First, lets first talk about broad categories of ML:-
- Supervised Learning: In this, the system is provided with clearly labelled data pairs of A-B. The ML model is then trained on this data so that it can establish causal relationship between A and B. Once perfected, the system can then predict A given B or vice-versa. It is commonly used for classification problems. One everyday application of SL is whether to classify an incoming email as spam or not spam.
2. Unsupervised Learning: As the name suggests- Here, the computer is given unlabelled data and then asked to self-identify inherent patterns and relationships, unlike SL were the training data is human-labelled in advance. It is useful because in many cases, obtaining labeled data is often difficult, costly, and/or time-consuming.
Its major application is Clustering. Cluster analysis is the process of grouping data in different segments based on similarities identified by ML model. A well-known example of clustering is in identifying fake news. The ML model examines the words used in an article and then clusters them which help determine which pieces are genuine and which are fake news.
3. Semi-Supervised Learning: In real world, most dataset contain noise, incorrect pairings, large number of un-labeled variables and a small set of well-labeled variables. Hence, Semi-supervised learning uses both labeled and un-labeled data to improve the accuracy of the learning model. Infact, most ML models in production environment use Semi-supervised technique. Common examples being recommendation systems, which group users on basis of their similar buying habits (items, music, movies) and prompt you with new suggestions.
4. Reinforcement Learning: This is learning from mistakes. Here, the system uses trial and errors to achieve a well-defined goal (maximize reward) in a dynamic environment. Influenced from the field of neuroscience and psychology, this ML system is built such that it associates good behaviour with a positive rewards and bad behaviour with negative, so that we can enforce the AI model to take “good” decisions, the good being defined by the programmer.
The algorithm (agent) evaluates a current situation (state), takes an action, and receives feedback (reward) from the environment after each act.
Reinforcement learning is most widely used technique of Machine Learning and now mostly employs deep learning models.
A day-to-day example: News Recommendation. Unlike items and movies etc which are clearly labelled by the platform itself (Amazon, Netflix for eg.), News articles are dynamic by nature and become rapidly irrelevant. User preferences in topics change as well. Here, RL provides better accuracy than SL or USL.
To understand RL better, take the case of ML-driven trading. Here, reinforcement learning models act in dynamic environment of stock market with a goal of making high profit (benchmarked against market index). The system is rewarded for every profitable trade and punished for a loss-making one. Through various trial and errors, the system learns to outperform the market and do better than human traders.
The modern day fields of Autonomous/self-driving vehicles and Robotics are powered by RL. Have you seen a Black & White movie restored in coloured format? That’s Deep Reinforcement Learning (DRL) at work for you. Infact, DRL has the capability to even generate original music.
Thats fine. But WT* is Deep Learning, Neural Networks and NLP !?
Of-course, every PM is curious to understand these hot-topics. From chatbots to Brain-computer interface, they are affecting nearly all Industries including those hitherto unaffected such as AI art gallery and drug discovery
Deep Learning is a sub-field of ML which deals with algorithms inspired by the structure and function of the brain. Hence, Artificial Neural Networks (ANN) are essentially the computing system/ architectures which help perform the act of deep learning. The neural networks are further categorised into deep neural networks, recurrent neural networks and convolution neural networks. But for our own sanity, let’s not get confused.
Deep Learning can be supervised, un-supervised or reinforced.
The DL models work in ANN layers. Each layer is assigned a specific portion of a task, and data might traverse the layers multiple times to refine and optimize the ultimate output. These “hidden” layers perform the algorithmic tasks that turn unlabelled, unstructured input into meaningful output.
The three layers are:-
- Input layer — input data for the neural network.
- Hidden layers — intermediate layer between input and output layer where all computation is performed.
- Output layer — produce the result for given inputs.
But Why Deep Learning? Simply because, DL models outperform traditional ML models in accuracy and prediction once a threshold data-input amount is reached.
The Key Differentiator between ML and DL is the concept of feature extraction (FE), the hardest concept to teach in field of AI to a novice.
Simply put, All real-world data consists of noise or redundant features. For example a photo of car also consists of other objects such as sky, tree, road, buildings etc in the background and even foreground.
DL models reduce all set of features to determine a subset of relevant features that contain the required information, so that the desired task can be performed by using this reduced representation instead of the complete initial data. Feature extraction is done by human in machine learning whereas deep learning model figure out by itself.
While deep learning models outperform other ML techniques, there are limitations such as they require large volume of data, significant computing power and robust training of data sets.
Finally, Deep Learning finds its applications in wide variety of fields including computer vision, machine vision, speech recognition, natural language processing(NLP), audio recognition among others.
One of the most popular usage areas of DL/NLP is voice search & voice-activated intelligent assistants. Xbox, Skype, Google Now and Apple’s Siri, are all employing DL technologies in their systems to recognize human speech and voice patterns.
Neural Networks, core component of DL, are central to NLP which helps computers to process and analyze large amount of human language data. They help improve machine translation, sentiment analysis and human retrieval. The famous example being Google Translate. They are also behind the ambitious long-term goal of chatbots- to replace humans completely and perform natural flow of human-like conversation.
Also, DL has revolutionised the field of computer vision, as DL based image recognition produce more accurate results than human contestants. It provides computer systems the ability to identify objects, places, people, writing and actions in images. The controversial example being of Face detection and surveillance.
And Predictions. DL models outperform ML in all possible kinds of predictions. From predicting earthquake to heart attack and stock prices, DL makes it possible to make quality decisions in dynamic environment.
The aim of this article was to provide a fundamental understanding of AI and the Jargon surrounding it. We intend this to be just a starting point for your long, adventurous journey in the field! Best of Luck!
If you enjoyed this post or found it helpful in any way, please feel free to applaud the article, using Medium Clap button (in left or below the post). The more you clap, farther the article reaches. To every Product Manager willing to up-skill.
Finally, in case of any questions, leave a comment.
And Stay Tuned!
This post has been published on www.productschool.com communities.
Don’t forget to give us your 👏 !
