Generated code description

Code description generated automatically from docstrings.

Models

Module for nets generation.

The three proposed designs are a small encoder (ToaEncoder) a time series LSTM (TimeSeriesLSTM) and a concatenation of the two (LstmEncoder).

class cloudatlas.nets.LinearProbe(lstmencoder, **net_kwargs)
class cloudatlas.nets.LstmEncoder(optimizer='adam', encoder=None, lstm=None, train_encoder=True, train_lstm=True, train_whole=True, **net_kwargs)

The net to analyze the AirShower dataset.

It is composed by a time of arrival branch and a time series branch.

The latter is designed as an encoder of dense layers. The hypothesis that brought to this design is the information redundancy of the time of arrival matrix. The more a paricle shower is homogeneous the less number of parameters are needed to describe it, such as an incidence angle, spread angle and height of first collision. The encoder aims to extract those “homogeneous beam” parameters.

The time series branch is composed of a layer of lstm units and a relu-activated dense layer. It processes the time evolution of the detectors activity.

Finally the output of the two branches are concatenated and porcessed with a small number of relu-activated dense layers. A final linear dense unit serves as the output.

Since the whole net’s purpose is a regression task the loss function is by default the mean squared error and the natural metric is the RMSE.

Parameters

  • optimizer (str or keras.optimizers.Optimizer) – the optimizer for the training stage. Default is Adam(lr=0.001).

  • net_kwargs (optional) – LushlooNet keyword args

  • encoder (keras.models.Model, optional) – the encoder sub-network. If nothing is given, a new empty encoder is created.

  • lstm (keras.models.Model) – the lstm sub-network. If nothing is given, a new empty lstm is created.

  • train_encoder (bool, optional) – specify whether to train or not the encoder, in case a pre trained one is given.

  • train_lstm (bool, optional) – specify whether to train or not the lstm, in case a pre trained one is given.

model

the (compiled) LstmEncoder network

Type

keras.models.Model

class cloudatlas.nets.LushlooNet(path='trained/LstmEncoder', tensorboard=False, earlystopping=False)

The basis class for other nets.

It has no real model definition, just shared utility methods.

Parameters

path (str) – the folder where to save the model.

path

the path of the folder.

Type

str

model
Type

keras.models.Model

tensorboard

specifies whether to use the TensorBoard callback

Type

bool, optional

earlystopping

specifies whether to use the EarlyStopping callback

Type

bool, optional

resolution_on(feeder)

Estimates the resolution on a specified dataset.

Parameters

feeder (DataFeeder) – the feeder of the dataset.

train(**fit_kwargs)

Trains the model and saves history.

class cloudatlas.nets.TimeSeriesLSTM(optimizer='adam', **net_kwargs)

The lstm net that processes time series matrices.

Parameters

  • optimizer (str or keras.optimizers.Optimizer) – the optimizer for the training stage. Default is Adam(lr=0.001).

  • net_kwargs (optional) – LushlooNet keyword args

class cloudatlas.nets.ToaEncoder(optimizer='adam', **net_kwargs)

The encoder that processes time of arrival matrices.

Parameters

  • path (str) – the folder where to save the model.

  • optimizer (str or keras.optimizers.Optimizer) – the optimizer for the training stage. Default is Adam(lr=0.001).

  • net_kwargs (optional) – LushlooNet keyword args

Feeders

Module for data feeders

class cloudatlas.datafeeders.DataFeeder(*args: Any, **kwargs: Any)

Data generator that uses a single-file split dataset of numpy structured arrays.

This means that the single record must have fields. Multiple inputs are supported.

Parameters

  • folder (str) – the folder where the dataset is stored.

  • input_fields (list or str) – the list of strings that specify the field names.

  • target_field (str) – the name of the target field.

  • batch_size (int, optional) – the batch size. Default is 32.

  • shuffle (bool, optional) – enable shuffling dataset on_epoch_end

data_generation(batch_datum_indexes)

Loads data and returns a batch

on_epoch_end()

Shuffles indexes after each epoch

class cloudatlas.datafeeders.FeederProf(*args: Any, **kwargs: Any)

Curriculum creator.

Takes a pre-trained model and estimates the ‘difficulty’ of each record. Then generates the batches with increasing difficulty.

Parameters

  • trained_model (keras.models.Model) – the trained model used to score data.

  • data_folder (str) – th folder of the dataset.

  • difficulty_levels (int, optional) – the total number of difficulty levels. Default 5.

  • n_of_epochs (int, optional) – the total number of epochs of the training for which the FeederProf is used. Default 20. The difficulty of the fed data starts from level 0 and increases linearly by a factor epoch/n_of_epochs.

on_epoch_end()

Generates the restricted dataset that will be used in the next epoch.

score_data()

Estimates the difficulty of the data.

Associates the indexes of the batch to a given difficulty score. This function must be called BEFORE the __getitem__ override, as it uses self as the generator.

Utilities

Utility module

class cloudatlas.utils.RemoteMonitor

Class for remote logging using telegram.

send(message)

Sends a message.

class cloudatlas.utils.RemoteStderr

A wrapper for sys.stderr

Output of errors is sent on telegram before being printed on screen

Augmentation