sparse – Symbolic Sparse Matrices [doc TODO]

The sparse module has been used in:

• NLP: Dense linear transformations of sparse vectors.
• Audio: Filterbank in Fourier domain.

The sparse module is less mature than the tensor module. This documentation is also not mature.

The sparse submodule is not loaded when we import theano. You must import theano.sparse to enable it.

The sparse module provides two kinds of sparse tensors are supported: CSC matrices and CSR matrices. Operations that are implemented:

grad?

• conversion from sparse <-> dense

  • theano.sparse.{dense_from_sparse,dense_from_sparse}

• [un]packing of sparse matrices from indexlists and nonzero elements.

  • packing: theano.sparse.{CRC,CSR}
  • unpacking: theano.sparse.csm_properties

• transpose

  • theano.sparse.transpose

• negation

  • neg

• addition/multiplication (elemwise)

  • theano.sparse.{add,mul}
  • sparse + sparse, sparse + dense, dense + sparse
  • sparse * sparse, sparse * dense, dense * sparse

• StructuredDot

  • with gradient defined such that sparsity pattern is constant. This function is called “structured_dot”
  • theano.sparse.structured_dot and its grad (structured_dot_grad)
  • theano.dot call it.
  • dot(sparse, dense) and dot(dense, sparse), dot(sparse, sparse)

• Dot

  • performs the true dot without special semantics.
  • dot(sparse, dense), dot(dense, sparse), dot(sparse, sparse)
  • When the operation has the form dot(csr_matrix, dense) the gradient of this operation can be performed inplace by UsmmCscDense. This leads to significant speed-ups.

• Subtensor

  • sparse_variable[N, N], return a tensor scalar
  • sparse_variable[M:N, O:P], return a sparse matrix
  • Don’t support [M, N:O] and [M:N, O] as we don’t support sparse vector and returning a sparse matrix would break the numpy interface. Use [M:M+1, N:O] and [M:N, O:O+1] instead.

There are no GPU implementations for sparse matrices implemented in Theano.

Some documentation for sparse has been written here.

sparse – Sparse Op

Platforms: Unix, Windows

Classes for handling sparse matrices.

To read about different sparse formats, see U{http://www-users.cs.umn.edu/~saad/software/SPARSKIT/paper.ps}.

@todo: Automatic methods for determining best sparse format?

class theano.sparse.basic.AddSD(use_c_code=True)

Add a sparse and a dense matrix

class theano.sparse.basic.AddSS(use_c_code=True)

Add two sparse matrices

class theano.sparse.basic.CSM(format, kmap=None)

Construct a CSC or CSR matrix from the internal representation

format

WRITEME

grad((data, indices, indptr, shape), (g_out, ))

Return a gradient on the data vector

kmap

WRITEME

make_node(data, indices, indptr, shape)

Build a SparseVariable from the internal parametrization

Parameters:

• data (1-d tensor) –
• indices (1-d tensor of ints) –
• indptr (1-d tensor of ints) –

perform(node, (data, indices, indptr, shape), (out, ))

Build a csc_matrix

class theano.sparse.basic.CSMProperties(kmap=None)

Extract all of .data .indices and .indptr

kmap

WRITEME

class theano.sparse.basic.DenseFromSparse(use_c_code=True)

Convert a sparse matrix to an ndarray.

sparse_grad

WRITEME

class theano.sparse.basic.Dot(use_c_code=True)

Operation for efficiently calculating the dot product when one or all operands is sparse. Supported format are CSC and CSR. The output of the operation is dense.

class theano.sparse.basic.GetItem2d(use_c_code=True)

Implement a subtensor of sparse variable and that return a sparse matrix.

If you want to take only one element of a sparse matrix see the class GetItemScalar that return a tensor scalar.

Note :that subtensor selection always returns a matrix so

indexing with [a:b, c:d] is forced. If one index is a scalar, e.g. x[a:b, c] and x[a, b:c], generate an error. Use instead x[a:b, c:c+1] and x[a:a+1, b:c].

The above indexing methods are not supported because the rval would be a sparse matrix rather than a sparse vector, which is a deviation from numpy indexing rule. This decision is made largely for keeping the consistency between numpy and theano. Subjected to modification when sparse vector is supported.

class theano.sparse.basic.GetItemScalar(use_c_code=True)

Implement a subtensor of a sparse variable that take two scalar as index and return a scalar

See :GetItem2d to return more then one element.

class theano.sparse.basic.MulSD(use_c_code=True)

Elementwise multiply a sparse and a ndarray

class theano.sparse.basic.MulSS(use_c_code=True)

Elementwise multiply a sparse and a sparse

class theano.sparse.basic.SparseType(format, dtype)

@type dtype: numpy dtype string such as ‘int64’ or ‘float64’ (among others) @type format: string @ivar format: The sparse storage strategy.

@note As far as I can tell, L{scipy.sparse} objects must be matrices, i.e. have dimension 2.

class theano.sparse.basic.StructuredDot(use_c_code=True)

Structured Dot is like dot, except that only the gradient wrt non-zero elements of the sparse matrix A are calculated and propagated.

The output is presumed to be a dense matrix, and is represented by a TensorType instance.

class theano.sparse.basic.Usmm(use_c_code=True)

Performs the expression is alpha * x y + z

x or y are sparse matrix(the other can be sparse or dense) z is a dense matrix alpha is a scalar

Note :We don’t implement the infer_shape as it is inserted by optimization only

class theano.sparse.basic.UsmmCscDense(inplace)

Performs the expression is alpha * x y + z This is an optimized operation for the case when x is in CSC format.

x are sparse matrix y, z is a dense matrix alpha is a scalar

Note :We don’t implement the infer_shape as it is inserted by optimization only

theano.sparse.basic.add(x, y)

Add two matrices, at least one of which is sparse.

theano.sparse.basic.as_sparse(x, name=None)

Wrapper around SparseVariable constructor. @param x: A sparse matrix. as_sparse_variable reads dtype and format properties out of this sparse matrix. @return: SparseVariable version of sp.

@todo Verify that sp is sufficiently sparse, and raise a warning if it is not

theano.sparse.basic.as_sparse_or_tensor_variable(x, name=None)

If we can’t make a sparse variable, we try to make a tensor variable.

theano.sparse.basic.as_sparse_variable(x, name=None)

Wrapper around SparseVariable constructor. @param x: A sparse matrix. as_sparse_variable reads dtype and format properties out of this sparse matrix. @return: SparseVariable version of sp.

@todo Verify that sp is sufficiently sparse, and raise a warning if it is not

theano.sparse.basic.dot(x, y)

Operation for efficiently calculating the dot product when one or all operands is sparse. Supported format are CSC and CSR. The output of the operation is dense.

theano.sparse.basic.mul(x, y)

Multiply (elementwise) two matrices, at least one of which is sparse.

theano.sparse.basic.structured_dot(x, y)

@todo: Maybe the triple-transposition formulation (when x is dense) is slow. See if there is a direct way to do this. (JB 20090528: Transposing tensors and sparse matrices is constant-time, inplace, and fast.)