I'm looking for software that can do symbolic matrix calculus with
vectors and matrices as first-class objects. That is, I'd like to be
able to define vectors and matrices by their sizes (and possibly other
properties) and be able to compute derivatives with the answer given
in terms of vectors/matrices rather than individual scalar elements.

Here are two interpretations of your question.  Answers given in Maple.

# interpretation 1:
restart;
A := < <a(t) | b(t)>, <c(t) | d(t)> >;     # A 2x2 matrix
map(diff, A, t);           # dA / dt
map(diff, A, t, t);        # d^2A / dt^2
map(diff, A.A, t);         # d(A^2) / dt

# interpretation 2:
restart;
diff(A(t).B(t), t);         # derivative of a non-commutative product
diff(A(t).B(t), t, t);      # second derivative of a non-commutative product

If you are asking for something else, you should be more explicit
in your description.

--
Rouben Rostamian

I'm looking for software that can do symbolic matrix calculus with
vectors and matrices as first-class objects. That is, I'd like to be
able to define vectors and matrices by their sizes (and possibly other
properties) and be able to compute derivatives with the answer given
in terms of vectors/matrices rather than individual scalar elements.

I guess I wasn't clear enough in my initial formulation of the
question. Let me try again.

Given the following (in Matlab notation):

a (n x 1 vector)
X (n x n matrix)
b (n x 1 vector)

I'd like to be able to compute derivatives like:

d(a'Xb)/dX = ab'

without ab' turning into [a1*b1 a1*b2 ; a2*b1 a2*b2]

This is obviously a very simple example, but I think the basic idea of
what I'm interested in should be clear. I'm mostly a Matlab user and
my Maple experience is largely via the Symbolic toolbox, but if
there's a way to do the above by issuing Maple commands that would be
great.

Given the following (in Matlab notation):

a (n x 1 vector)
X (n x n matrix)
b (n x 1 vector)

I'd like to be able to compute derivatives like:

d(a'Xb)/dX = ab'

without ab' turning into [a1*b1 a1*b2 ; a2*b1 a2*b2]

This is obviously a very simple example, but I think the basic idea of
what I'm interested in should be clear.

Given the following (in Matlab notation):
a (n x 1 vector)
X (n x n matrix)
b (n x 1 vector)
I'd like to be able to compute derivatives like:
d(a'Xb)/dX = ab'
without ab' turning into [a1*b1 a1*b2 ; a2*b1 a2*b2]
This is obviously a very simple example, but I think the basic idea of
what I'm interested in should be clear.

No, it isn't clear to me. It doesn't answer the question of
whether you want element-by-element computation or matrix computation.

If matrix solutions are considered, then your result is incorrect.
As X is allowed to vary, let X be a nilpotent matrix. Then Xb is n x n
of 0's, and a'Xb is 1 x n of 0's, for any a and b, which is clearly
not the same as ab' . (Does ab' even have the same size as d(a'Xb)/dX ??)

To be crystal clear, I'd
like to be able to compute the type of matrix derivatives that you can
find in Section 2 of the Matrix Cookbook (see Section 2.4.1 eq. 58 for
my example):

http://www.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf

I hope this makes things clear.

-Graham