tjruwase
Follow
Create your own GitHub profile
Sign up for your own profile on GitHub, the best place to host code, manage projects, and build software alongside 50 million developers.
Sign up
Popular repositories
165 contributions in the last year
Contribution activity
September 2020
Created a pull request in microsoft/DeepSpeed that received 1 comment
- Update ZeRO-Offload blog post link
- ZeRO tutorials
- ZeRO-Offload passing model tests
- ZeRO-Offload: Integration code fixes
- ZeRO-Offload passing model functionality tests
- Update installation instructions
- Generalize detection of ZeRO supported optimizers
- Assert ZeRO-Offload incompatible with gradient accumulation
- Add ZeRO-Offload checkpointing model tests
- guard 1bit adam reqs and update cond build for cpu-adam
- supporting different intermediate sizes other than 4 * hidden_dim
- Pipeline parallel training engine.
- fix zero-offload test with 16GB v100 nodes
- Fixing a link issue with SA tutorial
- Adding sparse attention news index item
- Minjiaz/zero offload
- fix cpu adam compilation for AVX2
- fix adam perormance
- fixing corner cases
- fixing adam copy fp16-param and add more compile flags for cpu_adam
- fixing the cpu_adam API and add deepspeed_adam flag in config.py
- Allocating CPU memory directly on CPU without transfering them from GPU
- add cpu adam optimizer
- Switches BBS example to use mbsize=3 and gas=2 to fit in 16GB of memory.