2021: A Tale of Three Networks
As we move into 2021, we will see increased integration of neural networks into software that runs our lives, enabling greater capabilities in spatial and sequential learning and making it tougher to differentiate between what is real and what is fake.
- By Troy Hiltbrand
- December 14, 2020
As we close out 2020, we have the opportunity to take stock of where we have been over the past year and look to the future and what is on the horizon. Although 2020 was a challenging year for society amid so much turmoil, we also saw an increased reliance on data and information. People learned just how important information was for making critical decisions in their lives. From models by some of the nation's leading institutions forecasting pandemic-related trends to the huge supercomputers allocated to run simulations on potential drugs and vaccines, analytics were everywhere this year.
What patterns do we anticipate continuing in 2021, and where do we see future developments? There are three technologies poised to become highly impactful in our lives: convolutional neural networks (CNN), recurrent neural networks (RNN), and generative adversarial networks (GAN). We have already seen great advances in neural networks, but 2021 will see these technologies incorporated further into commercial software offerings -- they will become a greater part of our everyday lives. The democratization of these highly complex technologies will be a major theme in the year ahead.
To understand what these neural networks can do, let's start by understanding what they are. Generally speaking, a neural network is a construct of machine learning and artificial intelligence that strives to simulate the neuron structure of the brain. It uses a sequence of interconnected nodes that leverage mathematical computations to transform and filter large amounts of data into patterns. These patterns are then applied to production data and the results drive decisions. Neural networks evolve based on the data provided to them during the training phase. This makes them flexible and able to adapt to different types of problem sets.
These three networks of the future all have the same basic structure but are constructed differently and are used to solve different types of problems. When employed across business use cases, they have the potential to change work as we know it. They will allow us to work more effectively and efficiently by allowing the computer to think at a higher level than any time in the past. As these technologies are integrated into our everyday software, these processes will also become more autonomous. They will allow our systems to process large amounts of data to find previously unrecognizable patterns and enable us to act on these patterns.
Convolutional Neural Network
A convolutional neural network (CNN) is a feed-forward artificial neural network. Using many levels of neurons, CNNs are structured so each successive layer in the network extracts additional features from the data. As the number of features grows, the model's ability to group and categorize input data increases. As CNNs grow, they consume large amounts of resources in the form of compute cycles and time.
CNNs are optimized for processing unstructured spatial data, such as images or video, and classifying or identifying this data. The area of computer vision (CV) relies heavily on CNNs. These convolutional neural networks are a key component in autonomous vehicle development and anomaly detection in images and videos such as cancer screening or automated quality assurance tasks.
As powerful as CNNs are, they do have a weakness: they are not adept at processing sequenced or time-based data. Their main function is to take in a large set of spatial data and extract and categorize it based on features.
As businesses deploy these technologies, CNNs with increased capacities will be used to automate problem sets that today require humans to think spatially and observe patterns visually.
Recurrent Neural Network
A recurrent neural network is different from a CNN in that it is not a stateless model. The connections between the nodes are structured so that information from later in the neural network easily flows back to previous parts of the model. This allows the RNN to learn from data sequences.
Because of the backwards connection of nodes within RNNs, the complexity of these networks grows exponentially with additional layers. Due to this complexity, these networks tend to remain much shallower than their CNN counterparts.
The ability to maintain state across the RNN makes it more optimal for problems that require data sequences and positional learning. This includes areas such as sentiment analysis in text processing, language translation, or time-based forecasting.
Problem sets that today require humans to think sequentially and see patterns between data points are driving improvement of RNNs. This includes tools that allow for time-series forecasting, such as algorithmic trading, revenue forecasting, and inventory movement management. Chatbots, language translation tools, and type-ahead technologies will also benefit from the utilization of these powerful neural networks.
Generative Adversarial Networks
A generative adversarial network is a construct where information can be synthetically generated. In a GAN, two neural networks are paired together to generate new data.
One neural network (the generator) is used to create data sets. It builds this data by taking real-world seed data and morphing it. The other network (the discriminator) attempts to determine if the data is real or has been synthetically generated by the generator. Over a cycle of training, the generator improves, creating data sets that appear more realistic, and in response the discriminator has to improve its ability to differentiate between real data and synthetic data.
GANs will be an ongoing hot topic because they can be used to synthetically create art, music, video, and literature that so very closely resembles its real counterpart that it is hard to differentiate between what's real and what's artificial.
Elon Musk's company, OpenAI, pushed improvements in GANs. With the release of GPT-3 this year, the world is seeing how possible it is for computers to create text content that is virtually indistinguishable from what humans can produce. They have also been able to demonstrate that with this technology, they can leverage a GAN to create music and art that mimics great artists of the past and present.
Through GANs, video can be created showing realistic representations of things that never happened. This can be leveraged to create videos of actors, politicians, and others without them ever participating. As you can guess, this is the potential dark side of neural networks. With the advent of deepfakes, society will be questioning how to perceive the difference between real life and artificially generated content. Companies are working to create discriminators that have the power to be increasingly sensitive in differentiating what is real and what is fake.
As we move into the future, we will see an increase in the sophistication of these GANs in both generating content and detecting generated content. We will see technology incorporated into the software that we use every day to help leverage this type of network and also combat its ill effects.
What's Next
Although the complexity surrounding these three types of networks is still high, they are becoming increasingly valuable to use cases across the board. As we proceed into 2021, we will see the deployment of more tools such as GPT-3 that increase access to these highly complex models.
We will also see the integration of these tools into commercial platforms, such as enterprise resource planning systems (ERP), customer relationship management systems (CRM), and marketing technology systems (MarTech). There will be a greater push to democratize this technology so that businesses will be able to incorporate these neural networks into their common business processes.
From these improvements will come much good, but unfortunately, we will also see challenges associated with inappropriate use of the same technology to harm society.
References
Chui, Michael; Kamalanth, Vishnu; and McCarthy, Brian. "An Executive's Guide to AI"
Choudary, Farhan and Linden, Alexander. "Innovation Tech Insight for Deep Learning," Gartner. 13 March 2020
SPRH Labs. "Understanding Deep Learning: DNN, RNN, LSTM, CNN and R-CNN," Medium, 21 Mar. 2019
Somers, Meredith. "Deepfakes, Explained," MIT Sloan, 21 July 2020