benchmarking

The original issue was reported in dotnet/performance#1701:

// Benchmark Process Environment Information:
// Runtime=.NET 6.0.0 (6.0.21.11801), X64 RyuJIT
// GC=Concurrent Workstation
// Job: Job-AQTFSE(PowerPlanMode=00000000-0000-0000-0000-000000000000, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)

OverheadJitting 

For different use cases, like bencheeorg/benchee_html#10 it'd be great to have statistics about statistics - what I'd call "meta statistics" - although there's probably some better real statistics name for this :)

What should be in there (that I know of so far):

• job size (how many jobs are in there)
• minimum of run times over all jobs
• maximum of run times over all jobs

This should be

For the moment, the nc_benchmark generator function allows for a nc_first_task option, which is good for pre-training in the class-incremental learning scenario. However, the same kind of option is not available if one wants to pretrain in the task-incremental scenario. It would be nice to have an option that could be used together with task_labels=True and allows for pretraining on multiple tasks

There seem to be some vulnerabilities in our code that might fail easily. I suggest adding more unit tests for the following:

• Custom agents (there's only VPG and PPO on CartPole-v0 as of now. We should preferably add more to cover discrete-offpolicy, continuous-offpolicy and continuous-onpolicy)
• Evaluation for the Bandits and Classical agents
• Testing of convergence of agents as proposed i