Basic functions using pytorch

Torch Core

This module contains all the basic functions we need in other modules of the fastai library (split with core that contains the ones not requiring pytorch). Its documentation can easily be skipped at a first read, unless you want to know what a given fuction does.

Global constants

AdamW = partial(optim.Adam, betas=(0.9,0.99))

bn_types = (nn.BatchNorm1d, nn.BatchNorm2d, nn.BatchNorm3d)

default_device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

Functions that operate conversions

flatten[source]

flatten(m)

Flattens all the layers of m.

model2half[source]

model2half(model:Module) → Module

Convert model to half precision except the batchnorm layers.

np2model_tensor[source]

np2model_tensor(a)

Tranform numpy array a to a tensor of the same type.

requires_grad[source]

requires_grad(m:Module, b:Optional[bool]=None) → Optional[bool]

If b is not set requires_grad on all params in m, else return requires_grad of first param.

tensor[source]

tensor(x:Any, rest) → Tensor

Like torch.as_tensor, but handle lists too, and can pass multiple vector elements directly.

to_cpu[source]

to_cpu(b:ItemsList)

Recursively map lists of tensors in b to the cpu.

to_data[source]

to_data(b:ItemsList)

Recursively map lists of items in b to their wrapped data.

to_detach[source]

to_detach(b:Tensors, cpu:bool=True)

Recursively detach lists of tensors in b; put them on the CPU if cpu=True.

to_device[source]

to_device(b:Tensors, device:device)

Recursively put b on device.

to_half[source]

to_half(b:Collection[Tensor]) → Collection[Tensor]

Set the input of batch b to half precision.

to_np[source]

to_np(x)

Convert a tensor to a numpy array.

try_int[source]

try_int(o:Any) → Any

Try to convert o to int, default to o if not possible.

Functions to deal with model initialization

apply_init[source]

apply_init(m, init_func:LayerFunc)

Initialize all non-batchnorm layers of m with init_func.

apply_leaf[source]

apply_leaf(m:Module, f:LayerFunc)

Apply f to children of m.

cond_init[source]

cond_init(m:Module, init_func:LayerFunc)

Initialize the non-batchnorm layers of m with init_func.

in_channels[source]

in_channels(m:Module) → List[int]

Return the shape of the first weight layer in m.

init_default[source]

init_default(m:Module, func:LayerFunc='kaiming_normal_')

Initialize m weights with func and set bias to 0.

Functions to get information of a model

children[source]

children(m:Module) → ModuleList

Get children of m.

first_layer[source]

first_layer(m:Module) → Module

Retrieve first layer in a module m.

last_layer[source]

last_layer(m:Module) → Module

Retrieve last layer in a module m.

num_children[source]

num_children(m:Module) → int

Get number of children modules in m.

one_param[source]

one_param(m:Module) → Tensor

Return the first parameter of m.

range_children[source]

range_children(m:Module) → Iterator[int]

Return iterator of len of children of m.

trainable_params[source]

trainable_params(m:Module) → ParamList

Return list of trainable params in m.

Functions to deal with BatchNorm layers

bn2float[source]

bn2float(module:Module) → Module

If module is batchnorm don't use half precision.

set_bn_eval[source]

set_bn_eval(m:Module)

Set bn layers in eval mode for all recursive children of m.

split_bn_bias[source]

split_bn_bias(layer_groups:ModuleList) → ModuleList

Split the layers in layer_groups into batchnorm (bn_types) and non-batchnorm groups.

Functions to get random tensors

log_uniform[source]

log_uniform(low, high, size:Optional[List[int]]=None) → FloatOrTensor

Draw 1 or shape=size random floats from uniform dist: min=log(low), max=log(high).

log_uniform(0.5,2,(8,))

tensor([0.5775, 0.7902, 0.6087, 0.5730, 0.8057, 0.8845, 0.8975, 0.5585])

rand_bool[source]

rand_bool(p:float, size:Optional[List[int]]=None) → BoolOrTensor

Draw 1 or shape=size random booleans (True occuring with probability p).

rand_bool(0.5, 8)

tensor([1, 1, 0, 1, 0, 0, 1, 0], dtype=torch.uint8)

uniform[source]

uniform(low:Number, high:Number=None, size:Optional[List[int]]=None) → FloatOrTensor

Draw 1 or shape=size random floats from uniform dist: min=low, max=high.

uniform(0,1,(8,))

tensor([0.6432, 0.3110, 0.7588, 0.7058, 0.7121, 0.8552, 0.3352, 0.2620])

uniform_int[source]

uniform_int(low:int, high:int, size:Optional[List[int]]=None) → IntOrTensor

Generate int or tensor size of ints between low and high (included).

uniform_int(0,2,(8,))

tensor([0, 1, 1, 2, 1, 1, 1, 2])

Other functions

class FloatItem[source]

FloatItem(obj) :: ItemBase

Basic class for float items.

calc_loss[source]

calc_loss(y_pred:Tensor, y_true:Tensor, loss_func:LossFunction)

Calculate loss between y_pred and y_true using loss_func.

data_collate[source]

data_collate(batch:ItemsList) → Tensor

Convert batch items to tensor data.

grab_idx[source]

grab_idx(x, i, batch_first:bool=True)

Grab the i-th batch in x, batch_first stating the batch dimension.

logit[source]

logit(x:Tensor) → Tensor

Logit of x, clamped to avoid inf.

logit_[source]

logit_(x:Tensor) → Tensor

Inplace logit of x, clamped to avoid inf

model_type[source]

model_type(dtype)

Return the torch type corresponding to dtype.

np_address[source]

np_address(x:ndarray) → int

Address of x in memory.

split_model[source]

split_model(model:Module, splits:Collection[Union[Module, ModuleList]], want_idxs:bool=False)

Split model according to the layers in splits.

If splits are layers, the model is split at those (not included) sequentially. If want_idxs is True, the corresponding indexes are returned. If splits are lists of layers, the model is split according to those.

split_model_idx[source]

split_model_idx(model:Module, idxs:Collection[int]) → ModuleList

Split model according to the indexes in idxs.

trange_of[source]

trange_of(x)

Create a tensor from range_of(x).