Instead of indexing to access the rows, you could use iteration:

l = sum(row[i] for i, row in enumerate(matrix))

Instead of doing two passes over matrix to calculate l and r, you could accumulate the difference in one pass:

difference = sum(row[i] - row[N-i-1] for i, row in enumerate(matrix))
return abs(difference)

Instead of indexing to access the rows, you could use iteration:

l = sum(row[i] for i, row in enumerate(matrix))

Instead of doing two passes over matrix to calculate l and r, you could accumulate the difference in one pass: (note also negative indexing from end of list)

difference = sum(row[i] - row[-i-1] for i, row in enumerate(matrix))
return abs(difference)

The single-pass approach would also allow you to process input line by line, instead of storing the whole matrix in memory.