layernorm.py 文件源码

python
阅读 24 收藏 0 点赞 0 评论 0

项目:chainer-speech-recognition 作者: musyoku 项目源码 文件源码
def forward(self, xs, eps=1e-6):
        self.retain_inputs(())
        self.eps = eps
        x = xs[0]
        self.x_shape = x.shape
        self.x_dtype = x.dtype
        xp = cuda.get_array_module(x)
        size = x.shape[1] * x.shape[2]
        self.x_size = size
        mean = xp.mean(x, axis=(1, 2), keepdims=True)
        self.broadcast_shape = mean.shape
        self.diff = x - mean
        std = xp.sqrt(xp.sum(self.diff ** 2, axis=(1, 2), keepdims=True) / size)
        # std = xp.std(x, axis=(1, 2), keepdims=True)
        self.std = std
        return self.diff / std,
评论列表
文章目录


问题


面经


文章

微信
公众号

扫码关注公众号