What are Neural Networks?

Artificial intelligence has three subfields that help it equip high-performance computers with powers that match and exceed human abilities. 

In this blog, you'll learn more about how the second AI subfield, neural networks, helps HPCs analyze data to increase situational awareness and ensure optimal performance across the modern battlespace. 

What are neural networks?

The second subset of artificial intelligence, neural networks are interconnected computing nodes that operate in a manner similar to neurons in the brain. 

Most commonly found in GPUs, they utilize algorithms to recognize hidden patterns and correlations in raw data, cluster and classify it, and, over time, continue to learn and improve from it. 

Originally, the purpose of neural networks was to act as a computational system, like the human brain, that could solve complex problems. Over time, neural networks have been used to match specific tasks like computer vision and speech recognition. 

Why are neural networks important?

Neural networks are suited to help people solve complex problems in real-life situations. 

These networks are able to learn and model the relationships between inputs and outputs that are nonlinear and complex, make generalizations and inferences, model highly volatile data, and reveal hidden relationships and patterns to predict unusual events. 

How do neural networks work?

A neural network consists of an input layer, an output (target) layer, and a hidden layer in between the two. Each layer consists of nodes. The layers are connected, and these connections form a network of interconnected nodes. 

A node is modeled after a neuron in the human brain. Much like neurons, nodes are activated when there is significant stimuli or input--in this case, oftentimes from the CPU. The activation spreads throughout the network, creating a response (output). 

The connections between the nodes act as synapses, enabling signals to be transmitted from one node to another. Signals are processed as they travel from the input layer to the output layer. 

When posed with a request or probe, the nodes run mathematical calculations to figure out if there's enough information to pass on to the next one. In other words, they read all the data and figure out where the strongest relationships exist. 

If data inputs are more than a certain threshold value, then the node "fires off" and activates the nodes that it is connected to. 

As the number of hidden layers within a neural network increases, deep learning neural networks are formed.

Deep learning takes simple neural networks to the next level. Using hidden layers, data scientists can build their own deep learning networks that enable machine learning. The computer can also learn on its own by recognizing patterns in many layers of processing.

What are the types of neutral networks? 

There are four different types of neural networks: convolutional neural networks (CNNs), recurrent neural networks (RNNs), feedforward neural networks, and autoencoder neural networks. 

1. Convolutional neural networks (CNNs): These contain five types of layers: input, convolution, pooling, fully connected, and output. Each layer has its own purpose, such as summarizing, connecting, or activating. CNNs have popularized image classification and object detection, and they have also been applied to other areas like natural language processing and forecasting.
2. Recurrent neural networks (RNNs): These use sequential information like time-stamped data from a sensor device or a spoken sentence, composed of a sequence of items. Unlike traditional neural networks, all inputs to an RNN are not independent of each other, and the output for each element depends on the computations of preceding elements. RMMs are used in forecasting, time series applications, sentiment analysis, and other text applications. 
3. Feedforward neural networks: These are networks in which each perceptron in one layer is connected to every perceptron from the next layer. Information is fed forward from one layer to the next in the forward direction only, and there are no feedback loops. 
4. Autoencoder neural networks: These are used to create abstractions called encoders, created from a given set of inputs. Though these are similar to traditional neural networks, these networks seek to model the inputs themselves, and therefore, the method is considered unsupervised. The primary purpose of autocoders is to desensitize the irrelevant and sensitize the relevant. As layers are added, further abstractions are formulated at higher layers--those closest to the point at which a decoder layer is introduced. These abstractions can be used by linear or nonlinear classifiers.

Source: druva.com. There are four different types of neural networks: convolutional neural networks (CNNs), recurrent neural networks (RNNs), feedforward neural networks, and autoencoder neural networks. 

What are the downsides of neural networks? 

Though neural networks can help enhance compute power, there are some drawbacks as well. 

1. The "Black Box" effect: Perhaps the most widely-known disadvantage of neural networks, this refers to when a neural network comes up with a certain output that does not seem to correlate with any inputs. For example, if you put an image of a rabbit into a neural network, and it predicts it to be a house, it's very hard to see how this conclusion was reached. Since interoperability is critical in many domains, the "black box" effect can pose serious problems, especially when dealing with customers. 
2. They take a long time to develop: When trying to solve a problem that no one has attempted before, you need more control over the machine learning algorithm. Although tools like Tenserflow may help, they can also make the process more complicated, and, depending on the project, development can take much longer. This raises the question of whether it is worth spending the time and money to develop something that can potentially be solved in a much simpler way. 
3. Massive amounts of data are needed: Typically, neural networks require much more data than traditional machine learning algorithms. This can be a difficult problem to deal with, since many ML problems can be solved with less data if you use other algorithms. Though there are exceptions, most neural networks perform poorly with little data. If possible, it is best to use a simpler algorithm that does not need a lot of data to solve problems.
4. Training them takes a lot of time and computational power: Deep learning algorithms that train really deep neural networks can take several weeks to train completely from scratch. In contrast, most machine learning algorithms take just a few minutes to a few days to train. In addition, the more layers of neural networks there are, the more compute power is needed to train them. 

Conclusion 

Neural networks are critical in enhancing situational awareness, recognizing objects and people, and strengthening surveillance and risk detection. 

These networks rely on inputs from CPUs to perform parallel processing within GPUs to analyze data in-real time and solve complex problems. 

Through interpreting data and extracting information, neural networks aid in helping autonomous vehicles, weapons, and drones detect risk to effectively track and engage with enemy threats, matching and exceeding human capabilities.

Our high-performance compute solutions are designed to improve the signal integrity between each component and, therefore, the speed at which AI can run is unmatched, both at the tactical edge and in climate-controlled environments. 

• "Neural Networks: What are they and why do they matter?" - SAS