jittor.init

这里是Jittor的参数初始化模块的API文档，您可以通过from jittor import init来获取该模块。

jittor.init.calculate_gain(nonlinearity, param=None)

Return the recommended gain value for the given nonlinearity function. The values are as follows:

nonlinearity	gain
Linear / Identity	\(1\)
Conv{1,2,3}D	\(1\)
Sigmoid	\(1\)
Tanh	\(\frac{5}{3}\)
ReLU	\(\sqrt{2}\)
Leaky Relu	\(\sqrt{\frac{2}{1 + \text{negative\_slope}^2}}\)
SELU	\(\frac{3}{4}\)

Args:

nonlinearity: the non-linear function (nn.functional name) param: optional parameter for the non-linear function

Examples:

>>> gain = nn.init.calculate_gain('leaky_relu', 0.2)  # leaky_relu with negative_slope=0.2

jittor.init.calculate_std(var, mode, nonlinearity, param=0.01)

jittor.init.constant(shape, dtype='float32', value=0.0)

Generate constant Jittor Var.

Args:

shape (int or tuple of int):

shape of the output Var

dtype (string):

dtype of the output Var, default float32

value (int or float):

value to be filled in output Var

Return:

A Jittor Var which filled by constant value.

Example:

from jittor import init
print(init.constant(2))
# output: [0.,0.]
print(init.constant((2,3), value=1.))
# output: [[1.,1.,1.],[1.,1.,1.]]

jittor.init.constant_(var, value=0.0)

Inplace initialize variable with constant value.

Args:

var (Jittor Var):

Var to initialize with constant value.

Return:

var itself.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.constant_(linear.weight)
print(linear.weight)
# output: [[0.,0.],[0.,0.]]
linear.weight.constant_() # This is ok too

jittor.init.eye(shape, dtype='float32')

Generate 2-D identity matrix.

Args:

shape (int or tuple of int):

shape of the output matrix

dtype (string):

dtype of the output matrix, default float32

Return:

A Jittor Var of identity matrix.

Example:

from jittor import init
print(init.eye(2))
# output: [[1.,0.],[0.,1.]]
print(init.eye((2,3), "float32"))
# output: [[1.,0.,0.],[0.,1.,0.]]

jittor.init.eye_(var)

Inplace initialize variable with identity matrix.

Args:

var (Jittor Var):

Var to initialize with identity matrix.

Return:

var itself.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.eye_(linear.weight)
print(linear.weight)
# output: [[1.,0.],[0.,1.]]
linear.weight.eye_() # This is ok too

jittor.init.fill(var, value=0.0)

Inplace initialize variable with constant value.

Args:

var (Jittor Var):

Var to initialize with constant value.

Return:

var itself.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.constant_(linear.weight)
print(linear.weight)
# output: [[0.,0.],[0.,0.]]
linear.weight.constant_() # This is ok too

jittor.init.gauss(shape, dtype='float32', mean=0.0, std=1.0)

Return Jittor Var initialize by random gauss.

Args:

shape (int or tuple of int):

shape of the output Var

dtype (string):

dtype of the output Var, default float32

mean (int or float or Var):

mean value of the random gauss

std (int or float or Var):

std value of the random gauss

Example:

from jittor import init
from jittor import nn
a = init.gauss((2,2), "float32", 0.0, 1.0)
print(a)

jittor.init.gauss_(var, mean=0.0, std=1.0)

Inplace initialize Jittor Var by random gauss.

Args:

var (Jittor Var):

Var to be initialized by random gauss

mean (int or float or Var):

mean value of the random gauss

std (int or float or Var):

std value of the random gauss

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.gauss_(linear.weight, 0.0, 1.0)
print(linear.weight)
linear.weight.gauss_(0.0, 1.0) # This is ok too

jittor.init.invariant_uniform(shape, dtype='float32', mode='fan_in')

Return Jittor initialized Var by invariant_uniform.

Args:

shape (int or tuple of int):

shape of the output Var

dtype (string):

dtype of the output Var, default float32

mode (string):

mode selection, should be fan_in or fan_out. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass.

Example:

from jittor import init
from jittor import nn
a = init.invariant_uniform_((2,2))
print(a)

jittor.init.invariant_uniform_(var, mode='fan_in')

Inplace initialize Jittor Var by invariant_uniform.

Args:

var (Jittor Var):

Var to be initialized by random invariant_uniform

mode (string):

mode selection, should be fan_in or fan_out. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.invariant_uniform_(linear.weight)
print(linear.weight)
linear.weight.invariant_uniform_() # This is ok too

jittor.init.kaiming_normal_(var, a=0, mode='fan_in', nonlinearity='leaky_relu')

Inplace initialize Jittor Var by kaiming_normal.

Args:

var (Jittor Var):

Var to be initialized by random kaiming_normal

a (float):

the negative slope of the rectifier used after this layer (only used with ‘leaky_relu’)

mode (string):

mode selection, should be fan_in or fan_out. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass.

nonlinearity (string):

nonlinearity used after this layer. It can be one of [linear, conv*, sigmoid, tanh, relu, leaky_relu]. leaky_relu is used by default.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.kaiming_normal_(linear.weight)
print(linear.weight)
linear.weight.kaiming_normal_() # This is ok too

jittor.init.kaiming_uniform_(var, a=0, mode='fan_in', nonlinearity='leaky_relu')

Inplace initialize Jittor Var by kaiming_uniform.

Args:

var (Jittor Var):

Var to be initialized by random kaiming_uniform

a (float):

the negative slope of the rectifier used after this layer (only used with ‘leaky_relu’)

mode (string):

mode selection, should be fan_in or fan_out. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass.

nonlinearity (string):

nonlinearity used after this layer. It can be one of [linear, conv*, sigmoid, tanh, relu, leaky_relu]. leaky_relu is used by default.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.kaiming_uniform_(linear.weight)
print(linear.weight)
linear.weight.kaiming_uniform_() # This is ok too

jittor.init.one(shape, dtype='float32')

Generate Jittor Var filled by one.

Args:

shape (int or tuple of int):

shape of the output Var

dtype (string):

dtype of the output Var, default float32

Return:

A Jittor Var which filled by one.

Example:

from jittor import init
print(init.one(2))
# output: [1.,1.]
print(init.one((2,3)))
# output: [[1.,1.,1.],[1.,1.,1.]]

jittor.init.one_(var)

Inplace initialize variable with one.

Args:

var (Jittor Var):

Var to initialize with one.

Return:

var itself.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.one_(linear.weight)
print(linear.weight)
# output: [[1.,1.],[1.,1.]]
linear.weight.one_() # This is ok too

jittor.init.relu_invariant_gauss(shape, dtype='float32', mode='fan_in')

Return Jittor Var initialized by relu_invariant_gauss.

Args:

shape (int or tuple of int):

shape of the output Var

dtype (string):

dtype of the output Var, default float32

mode (string):

mode selection, should be fan_in or fan_out. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass.

Example:

from jittor import init
from jittor import nn
a = init.relu_invariant_gauss((2,2))
print(a)

jittor.init.relu_invariant_gauss_(var, mode='fan_in')

Inplace initialize Jittor Var by relu_invariant_gauss.

Args:

var (Jittor Var):

Var to be initialized by random relu_invariant_gauss

mode (string):

mode selection, should be fan_in or fan_out. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.relu_invariant_gauss_(linear.weight)
print(linear.weight)
linear.weight.relu_invariant_gauss_() # This is ok too

jittor.init.trunc_normal_(var: jittor_core.jittor_core.Var, mean: float = 0.0, std: float = 1.0, a: float = - 2.0, b: float = 2.0) → jittor_core.jittor_core.Var

Fills the input jt.jittor_core.Var with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution \(\mathcal{N}(\text{mean}, \text{std}^2)\) with values outside \([a, b]\) redrawn until they are within the bounds. The method used for generating the random values works best when \(a \leq \text{mean} \leq b\). Args:

var: an n-dimensional jt.jittor_core.Var mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value

Examples:

from jittor import init from jittor import nn linear = nn.Linear(2,2) init.trunc_normal_(linear.weight, std=.02) print(linear.weight) linear.weight.trunc_normal_(std=.02) # This is ok too

jittor.init.uniform(shape, dtype='float32', low=0, high=1)

Generate random uniform Jittor Var.

Args:

shape (int or tuple of int):

shape of the output Var

dtype (string):

dtype of the output Var, default float32

low (int or float or Var):

lower bound value of the random uniform

high (int or float or Var):

upper bound value of the random uniform

Return:

A Jittor Var which filled by random uniform.

Example:

from jittor import init
print(init.uniform(5))
# output: [0.202268, 0.518688, 0.595274, 0.777354, 0.981979]
print(init.uniform((2,3), low=-1, high=1))
# output: [[ 0.6647397   0.2801202  -0.01981187]
#          [-0.9779438  -0.30149996  0.69056886]]

jittor.init.uniform_(var, low=0, high=1)

Inplace initialize Jittor Var by random uniform.

Args:

var (Jittor Var):

Var to be initialized by random uniform

low (int or float or Var):

lower bound value of the random uniform

high (int or float or Var):

upper bound value of the random uniform

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.uniform_(linear.weight, -1.0, 1.0)
print(linear.weight)
# output: [[ 0.6647397   0.2801202], [-0.9779438  -0.30149996]]
linear.weight.uniform_(-1.0, 1.0) # This is ok too

jittor.init.xavier_gauss(shape, dtype='float32', gain=1.0)

Return Jittor Var initialized by xavier_gauss, a.k.a xavier_normal.

The resulting var will have values sampled from \(gauss(-a, a)\) where

\[ext{std} = ext{gain} imes \sqrt{\]

rac{2}{ ext{fan_in} + ext{fan_out}}}

Args:

shape (int or tuple of int):

shape of the return Var.

dtype (string):

dtype of the return Var, default float32.

gain (float):

an optional scaling factor.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.xavier_gauss_(linear.weight, init.calculate_gain('relu'))
print(linear.weight)
linear.weight.xavier_gauss_() # This is ok too

jittor.init.xavier_gauss_(var, gain=1.0)

Inplace initialize Jittor Var by xavier_gauss, a.k.a xavier_normal.

The resulting var will have values sampled from \(gauss(-a, a)\) where

\[ext{std} = ext{gain} imes \sqrt{\]

rac{2}{ ext{fan_in} + ext{fan_out}}}

Args:

var (Jittor Var):

Var to be initialized by random xavier_gauss

gain (float):

an optional scaling factor.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.xavier_gauss_(linear.weight, init.calculate_gain('relu'))
print(linear.weight)
linear.weight.xavier_gauss_() # This is ok too

jittor.init.xavier_uniform(shape, dtype='float32', gain=1.0)

Inplace initialize Jittor Var by xavier_uniform.

The resulting var will have values sampled from \(uniform(-a, a)\) where

\[a = ext{gain} imes \sqrt{\]

rac{6}{ ext{fan_in} + ext{fan_out}}}

Args:

shape (int or tuple of int):

shape of the return Var.

dtype (string):

dtype of the return Var, default float32.

gain (float):

an optional scaling factor.

Example:

from jittor import init
from jittor import nn
a = init.xavier_uniform((2,2), gain=init.calculate_gain('relu'))
print(a)

jittor.init.xavier_uniform_(var, gain=1.0)

Inplace initialize Jittor Var by xavier_uniform.

The resulting var will have values sampled from \(uniform(-a, a)\) where

\[a = ext{gain} imes \sqrt{\]

rac{6}{ ext{fan_in} + ext{fan_out}}}

Args:

var (Jittor Var):

Var to be initialized by random xavier_uniform

gain (float):

an optional scaling factor.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.xavier_uniform_(linear.weight, init.calculate_gain('relu'))
print(linear.weight)
linear.weight.xavier_uniform_() # This is ok too

jittor.init.zero(shape, dtype='float32')

Generate zero Jittor Var.

Args:

shape (int or tuple of int):

shape of the output Var

dtype (string):

dtype of the output Var, default float32

Return:

A Jittor Var which filled by constant value.

Example:

from jittor import init
print(init.zero(2))
# output: [0.,0.]
print(init.zero((2,3)))
# output: [[0.,0.,0.],[0.,0.,0.]]

jittor.init.zero_(var)

Inplace initialize variable with zero.

Args:

var (Jittor Var):

Var to initialize with zero.

Return:

var itself.

Example:

from jittor import init
from jittor import nn
linear = nn.Linear(2,2)
init.zero_(linear.weight)
print(linear.weight)
# output: [[0.,0.],[0.,0.]]
linear.weight.zero_() # This is ok too