Hessian-based analysis is a technique used in machine learning to analyze the behavior of deep neural networks during training. Specifically, it involves computing the Hessian matrix, which is a matrix of second-order partial derivatives of the loss function with respect to the network’s parameters.
Large batch training refers to training deep neural networks using large batches of data instead of small ones. This approach has become increasingly popular due to its efficiency, but it can also lead to stability issues during training. Hessian-based analysis can be used to investigate these issues and determine if they are related to the curvature of the loss landscape.
Robustness to adversaries refers to a network’s ability to perform well even when faced with adversarial examples – inputs that have been intentionally modified to cause misclassification. Hessian-based analysis can also be used in this context, as it can provide insight into how changes in input affect the network’s output and help identify regions in input space that are particularly vulnerable to adversarial attacks.
Overall, Hessian-based analysis is a powerful tool for understanding how deep neural networks behave during training and how they respond to different types of inputs. It can help researchers develop more robust and efficient models by providing insights into the underlying mechanisms that govern their behavior.
