Query List with large number of items to perform aggregation

I'm not sure if this helps, since you are using caml (I have only really worked with MYSQL), but I will give you my explanation for that, and hopefully it will help your problem as well (I'm having a hard time finding references for the caml equivalent). What you need to do is include indexing (if CAML allows you to index your queries).

In MySQL, you have are able to run this query:

EXPLAIN SELECT col1,col2,col3 FROM table WHERE col1= 'val1';

This will return

+----------+------+---------------+------+---------+------+------+------------+
| table    | type | possible_keys | key  | key_len | ref  | rows | Extra      |
+----------+------+---------------+------+---------+------+------+------------+
| table    | ALL  | NULL          | NULL |    NULL | NULL |some# | where used |
+----------+------+---------------+------+---------+------+------+------------+

The NULL values in this example are qhy queries run through all possible combinations. If CAML has an equivilant to this, it will just show you keys and references used to query your database. You need to modify these keys to narrow down searches.

Unless you split apart your database into smaller tables, and then decide which tables to query code-side, this is the only option I know of.

If this answer isn't what you are looking for, let me know, and I'll remove it.