In Keras documentation, glorot_uniform says that the initializer is using Glorot Uniform from this paper. However, the Keras implementation is totally different from the equation on the paper. Also, there are some arguments such as mode ='fan_avg' is the default. It should be same as the referenced paper. 'fan_sum'. Golort uniform is shown in Equation 1, but Keras implementation is shown in Equation 2. I had hard time to produce the same or close results to Keras using Pytorch.

def __call__(self, shape, dtype=None): fan_in, fan_out = _compute_fans(shape) scale = self.scale if self.mode == 'fan_in': scale /= max(1., fan_in) elif self.mode == 'fan_out': scale /= max(1., fan_out) else: scale /= max(1., float(fan_in + fan_out) / 2) if self.distribution == 'normal': # 0.879... = scipy.stats.truncnorm.std(a=-2, b=2, loc=0., scale=1.) stddev = np.sqrt(scale) / .87962566103423978 x = K.truncated_normal(shape, 0., stddev, dtype=dtype, seed=self.seed) else: limit = np.sqrt(3. * scale) x = K.random_uniform(shape, -limit, limit, dtype=dtype, seed=self.seed) if self.seed is not None: self.seed += 1 return x