Tongping Liu

Is this project still alive?

3

I can't make it work even I strictly followed the guideline of README.

Questions about the Sharing of optimizer states

2

I noticed that ColossalAI provides few optimizers, such as 'FusedLAMB', 'FusedAdam', 'FusedSGD', 'Lamb', 'Lars', 'CPUAdam', 'HybridAdam'. These optimizers shards optimizer states based on the size of parameters and gradients. My...

**Describe the bug** After each step, when gradient_accumulation_steps is set to be 1 and in the end of each step, __reduce_and_partition_ipg_grads should be invoked to reduce the remainning gradients in...

Can't rollback successfully on Fluidanimate and kmeans

6

It is just stucked there in the rollback phase.

Currently, some semaphores are not cleaned up if we have to rollback. We add the thread.finalize() in finalize() function. However, this finalize() function is not called in the rollback phase....

Currently, pthread_cancel will invoke a commit at first. Then it will remove itself from the alive threads. We need to think about more on this.

Can't rollback successfully on ferret

3

It has some memory segmentation fault error

I don't know the reason why it can't rollback Dedup. It is just stucked somewhere in the rollback phase of Dedup.

Replaying of dining philosopher problem

1

We can't replaying the test case of dining philosopher problem: the program will be stucked there without any progress in the replaying phase. The stucked behavior can be changed by...

Developed the PoC of dAttention support. It will utilize the similar idea of vAttention, but it introduces a new memory layout that overcomes the waste of memory of vAttention.

1

FILL IN THE PR DESCRIPTION HERE FIX #xxxx (*link existing issues this PR will resolve*) **BEFORE SUBMITTING, PLEASE READ THE CHECKLIST BELOW AND FILL IN THE DESCRIPTION ABOVE** --- PR...