Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Chainer Benchmark
Benchmarking Chainer with Airspeed Velocity.
Requirements
You need to install asv benchmark framework and Cython (to build CuPy).
pip install asv CythonThe benchmark can be run over Chainer v4.0.0b1+.
Usage
To run one-shot benchmark against installed Chainer / CuPy:
asv run --python=same --show-stderrTo run benchmark to compare results between commits:
# Enable ccache for performance (optional).
export PATH="/usr/lib/ccache:${PATH}"
export NVCC="ccache nvcc"
# Run benchmark against target commit-ish of Chainer and CuPy.
# Note that specified versions must be a compatible combination.
# You can use `find_cupy_version.py` helper tool to get appropriate CuPy
# version/commit for the given Chainer version/commit.
./run.sh master master
./run.sh v4.0.0b4 v4.0.0b4
# Compare the benchmark results between two commits to see regression
# and/or performance improvements in command line.
alias git_commit='git show --format="%H"'
asv compare $(git_commit v4.0.0b4) $(git_commit master)
# Convert the results into HTML.
# The result will be in `html` directory.
asv publish
# Start the HTTP server to browse HTML.
asv previewAlternatively you can use Docker.
# Build docker image for benchmark.
docker build -t chainer-benchmark docker
# Create a machine configuration file (`.asv-machine.json`) in this directory (first time only).
nvidia-docker run --rm -u $(id -u):$(id -g) -v ${PWD}:/benchmarks -w /benchmarks -e HOME=/benchmarks chainer-benchmark asv machine --machine $(hostname)
# Run benchmark.
nvidia-docker run --rm -u $(id -u):$(id -g) -v ${PWD}:/benchmarks -w /benchmarks -e HOME=/benchmarks chainer-benchmark ./run.sh master master --machine $(hostname)