Tommy Mulc comments

Repositories
Issues
Comments

Results 3 comments of


                                            Tommy Mulc

Airflow Worker and Scheduler is not picking up job

I'm having the same issue, but only when using CeleryExecutor. LocalExecutor seems to work.

Improved Parameter Server performance

Would it be easier to use multiple parameters servers and shard the variables? You will eventually reach a bandwidth limit with one machine so it might be worth implementing this...

have you tryed training with the full dataset?

I did a non sanity check training (asynchronously on 384 cores) with lr=.00015, norm=ins, loss=l1, min_len=10, max_len=100, and r=5. Additionally, I used Luong Attention, gradient clipping by norm of 5,...