5

I was working on drafting/testing a technique I devised for solving differential equations for speed and efficiency.

It would require a storing, manipulating, resizing, and (at some point) probably diagonalizing very large sparse matrices. I would like to be able to have rows consisting of zeros and a few (say <5) ones, and add them a few at a time (on the order of the number of cpus being used.)

I thought it would be useful to have gpu accelleration--so any suggestions as to the best way to take advange of that would be appreciated too (say pycuda, theano, etc.)

Improve this question
4
  • How large of a matrix? Have you investigated numpy? Commented Aug 29, 2011 at 2:37
  • 1
    or try scipy.sparse? Commented Aug 29, 2011 at 2:40
  • I was anticipating something like this: stackoverflow.com/questions/1053928/… Commented Aug 29, 2011 at 2:52
  • I have used numpy and a bit of scipy. I mainly wanted to check that these were indeed the most scalable options--and also for any tips on using the gpu. Commented Aug 29, 2011 at 2:54

3 Answers 3

Reset to default
9

The most efficient storage method for symmetric sparse matrices is probably sparse skyline format (this is what Intel MKL uses, for example). AFAIK scipy.sparse doesn't contain a sparse, symmetric matrix format. Pysparse, however, does. Using Pysparse, you can build the matrix incrementally using the link list format, then convert the matrix into sparse skyline format. Performance wise, I have usually found Pysparse to be superior to scipy with large sparse systems, and all the basic building blocks (matrix product, eigenvalue solver, direct solver, iterative solver) as present, although the range of routines is perhaps a little smaller than what is available in scipy.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

5

You can use a dictionary and tuples to access the data:

>>> size = (4,4)
>>> mat = {}
>>> mat[0,1] = 3
>>> mat[2,3] = 5
>>> for i in range(size[0]):
        for j in range(size[1]):
            print mat.get((i,j), 0) ,
        print

0 3 0 0
0 0 0 0
0 0 0 5
0 0 0 0

Of course you should make a class for that and add the methods you need:

class Sparse(dict):
    pass

BTW, You can also use scipy.sparse from the scipy lib

Improve this answer

4 Comments

Ok, if that would be most efficient then so be it. Thank you, I mainly wanted to check for obscure answers.
I will just give the question another hour or so to see if any other answers pop up then I will mark this as solved. Thanks.
@Feynman A Sparse matrix class should overhide the __getitem__ method to return zero when there's no data
thanks for the tip. I am still pretty new to python. I will be sure to override the getitem method. I guess I will base the Sparse class on the scipy sparse class and modify some of the built in stuff.
3

Use scipy.sparse.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.