Fast Adaptation with Linearized Neural Networks

Abstract摘要

训练神经网络的归纳bias很难理解导致很难适应新任务。我们研究线性化神经网络的归纳bias，取得了全神经网络函数不错的结论。

我们提出了一项技术，通过来源于Jacobian of the network设计出来的kernal，将归纳bias embedding到高斯过程分布。

在这个设定中，domain adaptation的任务就有了基于不确定性估计可解释的后验推断。推荐是可分析的并且避免了局部最优，局部最优经常发生的fine-tuning神经网络权重到1个新的任务中。

我们开发了显著的计算提升通过矩阵相乘，Fisher vector products采用了一种新的实现。

develop significant computational speed-ups based on matrix multiplies, scalable Fisher vector products.

我们研究Linearizations of neural networks的归纳bias。the inductive biases of linearizations of neural networks, good summaries of the full netowrk functions.

embedding theses inductive biases into Gaussian processes through a kernel designed form the Jacobian of the network.

linearizations of neural networks 是什么？

good summaries of the full network functions 是什么？

Fisher vector products 是什么？

Jacobian of the network是什么？

Introduction 介绍

address the limitations of finetuning with minimal computational overhead

while keeping the performance competitive with costly and involved solutions based on adapting the full network.

Firstly. we linearize the DNN with a first order Taylor expansion, giving rise to a linear model whose inductive biases we study here empirically.

首先，将DNN通过一阶泰勒展开式，使其变成一个线性的模型，经验性的。

Secondly, we embed these inductive biases into a probabilistic lightweight framework which takes the form of a Bayesian linear model with the DNN Jacobian matrix J as the features.