Now that we have had some time to look at the regular neural networks, it is time for us to dive a bit deeper and look at another option, known as recurrent neural networks. These are going to follow some of the same rules that we can see with the discussion above, but they can also take some of your projects to the next level. Let’s take a look at what these recurrent neural networks are like, and how they can really benefit a project that you are working with. See more from What is Neural Networks? or IBM
A good way to start looking at these recurrent neural networks, or RNN, is by taking a look at the human brain. When we do this, we know that it is reasonable to have the understanding that our thought processes or our understanding don’t restart every second. We are able to retain the information that we hear and learn and then build on it. This is something that we do from childhood. We don’t just see the letter A and then forget about it five seconds later. We use it as the start of the alphabet and build on that to B, C and so on. We are always building on new knowledge, whether it is from our childhood, or if it is something that we just learned.
As we go through some of the other parts that show up in this guidebook, you will start to see how each of the words, based on how much understanding you had of the words that we wrote on the page before. Your brain is not going to see a word and then immediately threw it away, and then restart its thinking process from the beginning. The point here is that our thoughts are basically able to have some consistency and some persistence with them, which is part of what makes them so powerful to work with.
The traditional neural network that we discussed in the last section isn’t capable of doing this kind of thing. And this can be a bigger shortcoming in many cases. For example, if you are working on a project that needs to classify the kind of event that is happening during all of the different parts of a movie, it wouldn’t help you much with a traditional neural network because it would not be able to reason with the events that occurred earlier on in this film. There just isn’t that kind of communication or power showing up within the program.
Recurrent Neural Networks
Let’s take a look at the chart above. Each of the rectangles that are there is going to be a vector and the arrows are going to show us the functions. The input vectors are going to show up in red, and then the output vectors that we need to know are going to be in blue. And then the green vectors will hold onto the RNN state (which we are going to talk about in a minute). Going from the leftover to the right, let’s take a look at how each of these words:
- The first one is going to be the vanilla mode of processing, the one that doesn’t use the RNN at all. This is going to include an input that is fixed and an output that is fixed. This is also known as image classification.
- The sequence output is going to be the second part. This is going to be image captioning that is able to take an image and then will provide you with an output of a sentence of words.
- Sequence input: This is going to be the third picture above. It is going to be more of a sentiment analysis that shows us a given sentence and makes sure that it is classified as either a negative or positive sentiment.
- Sequence output and sequence output. You can find this one in the fourth box, and it is getting a bit closer to what we want. This one is going to be similar to a machine translation. This is when the RNN is able to read a sentence out in English, and then can take that information and provide you with an output that reads the sentence in French.
- And finally, the last box is going to be the synced sequence input and output. The video classification here is going to help us to label out each of the frames that occur in a video if we decide to.
Notice that in each of these, there isn’t going to be any constraints put on the lengths of the sequences that we have to specify ahead of time. this is because the recurrent transformation, which is going to be shown in green, is fixed, and we are able to apply it out as many times as we would like, or as many times as work with our project.
These neural networks are a great addition to any of the codes that you would like to write out in the Python language, and they are definitely good examples of what we are able to work with when it comes to deep learning. Deep learning can lead us to a lot of different projects, many of which we may not think were possible in the past. But with some great neural networks, including the recurrent networks and the traditional neural networks, we are able to make these programs happen, search through a lot of information, and get the predictions and insights that we are looking for out of that data.