Optmizing a function scipy.optimize

Question

I am new to python and statistical programming. For a class assignment we've been asked to implement python lasso L1 regression. This involves using a QP solver to solve .

0.5 *(x^t * H * x) + f^t * H
st x > 0 (every element of x is greater than zero)

These are block vectors and matrices. I am using 2 dimensional arrays for vectors and four dimensional array for the matrix H

def function(x):
    x = x.reshape(2, -1)
    return 0.5*np.tensordot(x,(np.tensordot(H,x,([1,3],[0,1]))),([0,1],[0,1])) + np.tensordot(f,x,([0,1],[0,1]))

initial_val = np.random.randn(2 * (k+1)).reshape((2,k+1))

bnds = (0,None)
theta = scipy.optimize.minimize(function, initial_val, method="SLSQP", bounds=bnds)

But still I am getting negative values in theta.x vector. Could anyone tell me Where am I going wrong?

You'll need to provide some more information, namely what you called function and initial_val, the error might lie with those. — Hooked
– Hooked, Commented Nov 18, 2013 at 15:24
I think x0 (initial_val) needs to be a vector, not a matrix. bnds needs to be of the same length as initial_val. It's difficult to reproduce your error without examples of H, f, and k. — Rob Falck
– Rob Falck, Commented Nov 19, 2013 at 14:06

BartoszKP · Accepted Answer · 2013-11-18 15:30:04Z

1

You need to provide constraints for the optimization, like this if you optimize a scalar function:

scipy.optimize.minimize(
    function,
    initial_val,
    method="SLSQP",
    bounds=bnds,
    constraints = [{'type':'ineq', 'fun':lambda x: x}])

or for vector functions:

constraints = [{'type':'ineq', 'fun':lambda x: x[i]} \
                   for i in range(len(initial_val))]

Note that if it's a vector, then you also need to provide bounds for each element:

 bnds = [(0, None) for _ in range(len(initial_val))]

You might also want to look at the reference.

answered Nov 18, 2013 at 15:30

BartoszKP

36k15 gold badges109 silver badges135 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Ada Xu Over a year ago

Although I got the result I was looking for I am still getting negative values in x vector after optimization. I used constraints = [{'type' : 'ineq', 'fun' : lambda x: x[i]} for i in range(len(initial))] . I can't understand why

BartoszKP Over a year ago

@AdaXu Have you verified that the optimization was successful (result.success and result.message attributes)? If yes, what do you mean that you got the result you were looking for and x had negative elements? I thought negative elements were not the result you were looking for.

Ada Xu Over a year ago

result.message - Optimization terminated successfully and the regression curve fits well. But there are negative values in result.x I don't know what to make of this

BartoszKP Over a year ago

@AdaXu Could you paste here the line with the minimize function call, and the actual values of result.x? Or maybe into pastebin? If I won't be able to help right away I'd suggest another question, but maybe it's a quick thing.

Ada Xu Over a year ago

I figured it out. The vector argument initial_val is actually a 2 dimension numpy array since my objective function involves block matrix H with 4 dimensions and block vectors with two dimensions. I changed them to ordinary 2 dimensional matrix and one dim vector and I get the right answer. If you can modify your answer for 2 dim constraint I will mark it as correct. Thanks for the help :)

|

Collectives™ on Stack Overflow

Optmizing a function scipy.optimize

1 Answer 1

6 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Related