boone.training

Class RMSpropTrainer

java.lang.Object

boone.Trainer

boone.training.BackpropTrainer

boone.training.RMSpropTrainer

All Implemented Interfaces:

Storable, java.lang.Cloneable

public class RMSpropTrainer
extends BackpropTrainer

The Root Mean Square propagation trainer. Instead of directly using the gradient information, a specific weight update value is calculated for each weight (and bias) based on the current gradient and the the previous gradient of that weight. Both gradients are squared, and a weighted sum of both is generated which acts as a regularization term of the weight update. Hence, the update values, i.e., weight changes, adapt to the training progress. Note that it may still be beneficial to optimize the initial learn rate for a specific problem. This method seems to work well with mini-batches, but has some problems in batch mode.

Since:

June 2019

Version:

$Id$

Author:

Helmut A. Mayer

Field Summary

Fields inherited from class boone.Trainer

currentEpoch, epochs, learnRate, net, shuffle, stepMode, testData, trainingData, trainingSignalGenerator

Constructor Summary

Constructors

Constructor and Description
RMSpropTrainer() Creates an RMSprop trainer with a learn rate of 0.001, a batch size of 32.

Method Summary

Modifier and Type	Method and Description
protected double	calcAdaptation(BrainPart part) Calculates the RMSprop adaptation using a moving average of the squared gradient.
void	fromXML(org.jdom2.Element element) Reads attributes from XML.
double	getEpsilon() Returns the fuzz factor used to avoid numerical problems.
double	getGamma() Returns the value of the weight of the current squared gradient.
void	setEpsilon(double epsilon) Returns the fuzz factor used to avoid numerical problems.
void	setGamma(double gamma) Sets the value of the weight of the current squared gradient.
org.jdom2.Element	toXML(org.jdom2.Element element) Writes specific elements to XML tree.

Methods inherited from class boone.training.BackpropTrainer

endPattern, endTrain, reset, train, train, train

Methods inherited from class boone.Trainer

clone, endBatch, getBatchSize, getEpochs, getLearnRate, getNetwork, getRank, getTestData, getTrainingData, getTrainingSignalGenerator, getWinningNeuron, setBatchSize, setEpochs, setLearnRate, setNetwork, setShuffle, setStepMode, setTestData, setTrainingData, setTrainingSignalGenerator, test, test, toString, train, trainPattern, updateLinks, updateNeurons

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

RMSpropTrainer

public RMSpropTrainer()

Creates an RMSprop trainer with a learn rate of 0.001, a batch size of 32.

Method Detail

calcAdaptation

protected double calcAdaptation(BrainPart part)

Calculates the RMSprop adaptation using a moving average of the squared gradient.

Overrides:

calcAdaptation in class BackpropTrainer

Parameters:

part - the brain part to be adapted

Returns:

the RMSprop adaptation value

getGamma

public double getGamma()

Returns the value of the weight of the current squared gradient. The weight of the previous squared gradients is then (1 - weight), so the weight value has to be in the unit interval.

Returns:

weight value of current squared gradient

setGamma

public void setGamma(double gamma)

Sets the value of the weight of the current squared gradient.

Parameters:

gamma - weight value (in unit interval)

getEpsilon

public double getEpsilon()

Returns the fuzz factor used to avoid numerical problems.

Returns:

the fuzz factor value

setEpsilon

public void setEpsilon(double epsilon)

Returns the fuzz factor used to avoid numerical problems.

Parameters:

epsilon - the fuzz factor value

toXML

public org.jdom2.Element toXML(org.jdom2.Element element)

Writes specific elements to XML tree.

Specified by:

toXML in interface Storable

Overrides:

toXML in class Trainer

Parameters:

element - the parent element

Returns:

the trainer element

fromXML

public void fromXML(org.jdom2.Element element)

Reads attributes from XML.

Specified by:

fromXML in interface Storable

Overrides:

fromXML in class Trainer

Parameters:

element - a JDOM element