twitter
Rewrite it all on Python
Rust code sucks for an open source project like this, I recommend rewriting it in Python ( I am 100% dead serious) It's already trying to use PyTorch so might as well go full Python.
You have until 12am EST to rewrite the whole codebase in Python. Go get 'em champ.
Switch from rust to an inferior bytecode-compiling interpreted language? How do you think the pipeline described runs approximately 5 billion times per day and completes in under 1.5 seconds on average? And you think switching to Python will make that faster? Lol.
Switch from rust to an inferior bytecode-compiling interpreted language? How do you think the pipeline described runs approximately 5 billion times per day and completes in under 1.5 seconds on average? And you think switching to Python will make that faster? Lol.
To be fair, my man never talked about making anything any faster at all here. In fact, I believe he thinks making it run 10000000x slower would end up being better, just think about it:
If users get the best recommendations possible at every second due to this being run multiple times throughout the day, that means that this whole system is completely ignorant on basic psychology. Let me expand on this, there's something called Delayed Gratification, as well as Instant Gratification, currently twitter works its users with the latest, giving them exactly what they want to see at the moment, but in the long run this makes it so people leave twitter at an earlier time, because they found too much gratification in a short while and they'd rather look for something that sort of challenges them.
Now, objectively speaking, the longer you retain your users, the better, and how to achieve that? Well, just throw some bad recommendations along with the good ones. So I'm 100% certain that babuloseo's proposal is the way to go here. You ask why? Let's see, rewriting the entire codebase in Python would imply:
- Throwing away this entire codebase that's probably been worked on by 100s of engineers for more than a decade, and having it rewritten by some foreign developers from Fiverr that would work for 5$/hr, ending up with an algorithm that's probably not as good.
- It would probably be required to be delivered by the end of April 1st, so not a lot of time to actually validate whatever was done
- It would take 10000000x time to compile and train
The end result? This but much worse, which will probably make Twitter more money for what I explained earlier, making finding the good stuff more difficult would make users more engaged with some features of the platform, making their stays longer.
This is how CEOs are chosen, not based in the faster solution, but in the better one.
Work smarter not faster.
-- I'm actually half serious, I laughed my ass off writing that but in case it made sense to some, it's because it does make sense
