uk.ac.gla.dcs.renaissance.iqir
Class DensityTracker

java.lang.Object
  bpiwowar.maths.eigen.IncrementalSymmetricED
      uk.ac.gla.dcs.renaissance.iqir.DensityTracker

All Implemented Interfaces:

Serializable

@Persistent
public class DensityTracker
extends bpiwowar.maths.eigen.IncrementalSymmetricED
implements Serializable

Track subspaces defined by a set of vectors. It is based on rank one updates of an eigenvalue decomposition, and deals with the sparsity of vectors

In the case of a mixture, computes the sum of alpha(i) * rho(i) for each component of the mixture

Author:

Benjamin Piwowarski

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class bpiwowar.maths.eigen.IncrementalSymmetricED

bpiwowar.maths.eigen.IncrementalSymmetricED.ChangeListener

Constructor Summary

DensityTracker()

DensityTracker(it.unimi.dsi.fastutil.ints.Int2IntSortedMap map, boolean updateMap)

Method Summary

void	add(double weight, bpiwowar.maths.matrix.DoubleMatrix1D weightedVector) Add a set of words In practice, compute a rank-one update of weight^2 x weightedVector
void	add(double sigma, bpiwowar.maths.matrix.DoubleMatrix1D u, int[] localToGlobalMap) Add a sigma * u to the column space (that is, update by sigma^2 * u * u^T), using the map to go from one id to another.
void	add(bpiwowar.maths.matrix.DoubleMatrix1D weightedVector) Add a set of words
void	add(EigenDecomposition doc) Add another sparse EVD
void	addLocal(double rho, bpiwowar.maths.matrix.DenseDoubleMatrix1D weightedLocalVector) Add a dense vector (i.e.
void	addLocal(double rho, bpiwowar.maths.matrix.DenseDoubleMatrix1D weightedLocalVector, boolean allowCompaction) Add a dense vector (i.e.
void	checkCompaction() Check & compact
bpiwowar.maths.matrix.DiagonalDoubleMatrix	computeSqrtSigma() Returns sqrt(S)
int[]	getLocalToGlobalTermIdMap() Return the reverse map
it.unimi.dsi.fastutil.ints.Int2IntSortedMap	getMap()
int	getNumberOfTerms()
int	getNumberOfUpdates()
bpiwowar.maths.matrix.DenseDoubleMatrix1D	getSum()
int	getUpdatesSinceLastCompaction() Get the number of updates since the last compaction
double	getWeightsSum()
void	rankOneUpdate(double rho, bpiwowar.maths.matrix.DoubleMatrix1D u, int[] localToGlobalMap) Add a rho * uu^T to the column space, using the map to go from one id to another.
void	rankOneUpdate(double rho, bpiwowar.maths.matrix.DoubleMatrix1D u, int[] localToGlobalMap, boolean allowCompaction) Add a rho * uu^T to the column space, using the map to go from one id to another.
void	removeMap() When using only a dense term space, this is useful to prevent bugs (we crash the program directly)
void	setCompaction(int compactInterval, double compactThreshold) Set the compaction interval (i.e.
int	setNumberOfUpdates(int numberOfUpdates)
void	setRealSelector(bpiwowar.maths.eigen.selector.Selector selector) Set the selector without introducing the default non zero selector as with setSelector
void	setSelector(bpiwowar.maths.eigen.selector.Selector selector) Set the selector for this tracker -- we add up our own non zero selector
void	setStoreVectorSum(boolean b) Should we store the sum of vectors?
boolean	trim(boolean trimMatrices, double threshold, boolean reorthogonalise) Reduce as much as possible the space taken by the representation, by removing terms which are not needed anymore (their row in the SVD decomposition is 0), and by adjusting the matrix size if needed
void	updateMap(int[] list) Update our global to local map with another map (i.e., the global ids of the other map will be known by ours)
void	updateMap(it.unimi.dsi.fastutil.ints.Int2IntMap otherMap) Update our global to local map with another map (i.e., the global ids of the other map will be known by ours)

Methods inherited from class bpiwowar.maths.eigen.IncrementalSymmetricED

computeU, computeU, getRank, getSelector, getSigma, getSigma, getU1, getU2, removeRows, reorthogonalise, setBrokenArrowSVD, setFactories, setFactory, setRank, trimMatrices, update

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DensityTracker

public DensityTracker()

DensityTracker

public DensityTracker(it.unimi.dsi.fastutil.ints.Int2IntSortedMap map,
                      boolean updateMap)

Method Detail

add

public void add(bpiwowar.maths.matrix.DoubleMatrix1D weightedVector)

Add a set of words

Parameters:

weightedVector - The word vector within the global term id space

add

public void add(double weight,
                bpiwowar.maths.matrix.DoubleMatrix1D weightedVector)

Add a set of words In practice, compute a rank-one update of weight^2 x weightedVector

Parameters:

weight - The global weight for the vector

weightedVector - The word vector within the global term id space

addLocal

public void addLocal(double rho,
                     bpiwowar.maths.matrix.DenseDoubleMatrix1D weightedLocalVector)

Add a dense vector (i.e. already in the local term id space)

Parameters:

weightedLocalVector - The vector to add

addLocal

public void addLocal(double rho,
                     bpiwowar.maths.matrix.DenseDoubleMatrix1D weightedLocalVector,
                     boolean allowCompaction)

Add a dense vector (i.e. already in the local term id space), i.e. add rho * a * a^T

Parameters:

weightedLocalVector - The vector to add

allowCompaction - If true, allows compaction after the update (compaction might change the map)

checkCompaction

public void checkCompaction()

Check & compact

add

public void add(double sigma,
                bpiwowar.maths.matrix.DoubleMatrix1D u,
                int[] localToGlobalMap)

Add a sigma * u to the column space (that is, update by sigma^2 * u * u^T), using the map to go from one id to another. Warning: for efficiency reasons, the map of this subspace tracker must have been updated with #updateMap(Int2IntOpenHashMap)

Parameters:

sigma - The sigma for the u dimension

u - The vector to add

localToGlobalMap - The map to convert a local index (in u) to a global word id

rankOneUpdate

public void rankOneUpdate(double rho,
                          bpiwowar.maths.matrix.DoubleMatrix1D u,
                          int[] localToGlobalMap)

Add a rho * uu^T to the column space, using the map to go from one id to another. Warning: for efficiency reasons, the map of this subspace tracker must have been updated with #updateMap(Int2IntOpenHashMap)

Parameters:

rho - The sigma for the u dimension

u - The vector to add

localToGlobalMap - The map to convert a local index (in u) to a global word id

rankOneUpdate

public void rankOneUpdate(double rho,
                          bpiwowar.maths.matrix.DoubleMatrix1D u,
                          int[] localToGlobalMap,
                          boolean allowCompaction)

Add a rho * uu^T to the column space, using the map to go from one id to another. Warning: for efficiency reasons, the map of this subspace tracker must have been updated with #updateMap(Int2IntOpenHashMap)

Parameters:

rho - The sigma for the u dimension

u - The vector to add

localToGlobalMap - The map to convert a local index (in u) to a global word id

allowCompaction - If we can sparsify after the update

getLocalToGlobalTermIdMap

public int[] getLocalToGlobalTermIdMap()

Return the reverse map

Parameters:

tracker -

Returns:

updateMap

public void updateMap(it.unimi.dsi.fastutil.ints.Int2IntMap otherMap)

Update our global to local map with another map (i.e., the global ids of the other map will be known by ours)

Parameters:

otherMap - The other map to include

updateMap

public void updateMap(int[] list)

Update our global to local map with another map (i.e., the global ids of the other map will be known by ours)

Parameters:

list - The list of global term ids to add

getNumberOfTerms

public int getNumberOfTerms()

trim

public boolean trim(boolean trimMatrices,
                    double threshold,
                    boolean reorthogonalise)

Reduce as much as possible the space taken by the representation, by removing terms which are not needed anymore (their row in the SVD decomposition is 0), and by adjusting the matrix size if needed

Parameters:

trimMatrices - True if the matrices of the SVD decomposition should be trimmed after the cleaning process

threshold - The threshold for a row norm below which the row is removed

reorthogonalise - true if the basis should be reorthogonalised in case there are some changes

Returns:

true if there was a dimension reduction

setSelector

public void setSelector(bpiwowar.maths.eigen.selector.Selector selector)

Set the selector for this tracker -- we add up our own non zero selector

Overrides:

setSelector in class bpiwowar.maths.eigen.IncrementalSymmetricED

setRealSelector

public void setRealSelector(bpiwowar.maths.eigen.selector.Selector selector)

Set the selector without introducing the default non zero selector as with setSelector

Parameters:

selector - The selector for eigenvalues

setCompaction

public void setCompaction(int compactInterval,
                          double compactThreshold)

Set the compaction interval (i.e. number of vectors added before we start compacting the structures)

Parameters:

compactInterval -

setStoreVectorSum

public void setStoreVectorSum(boolean b)

Should we store the sum of vectors?

Parameters:

b - True to store, false to not store

getSum

public bpiwowar.maths.matrix.DenseDoubleMatrix1D getSum()

Returns:

the sum

getMap

public it.unimi.dsi.fastutil.ints.Int2IntSortedMap getMap()

computeSqrtSigma

public bpiwowar.maths.matrix.DiagonalDoubleMatrix computeSqrtSigma()

Returns sqrt(S)

setNumberOfUpdates

public final int setNumberOfUpdates(int numberOfUpdates)

Parameters:

numberOfUpdates - the numberOfUpdates to set

getNumberOfUpdates

public int getNumberOfUpdates()

Returns:

the numberOfUpdates

add

public void add(EigenDecomposition doc)

Add another sparse EVD

Parameters:

doc -

getUpdatesSinceLastCompaction

public int getUpdatesSinceLastCompaction()

Get the number of updates since the last compaction

Returns:

the updatesSinceLastCompaction

getWeightsSum

public double getWeightsSum()

removeMap

public void removeMap()

When using only a dense term space, this is useful to prevent bugs (we crash the program directly)