xiscoding

Change ints to double in quant_cuda_kernel.cu?

6

I was getting this error when running python setup_cuda.py quant_cuda_kernel.cu(149):` error: no instance of overloaded function "atomicAdd" matches the argument list argument types are: (double *, double) detected during instantiation...

Token generation limit

5

I was getting the RateLimitReached Error yesterday (after around the 80th generation each prompt is around 10000 tokens). My simple workaround is below, but is there a better way? ```...