Machine Learning, AI, NLP, Programming, Deep Dives.
the day, if you are only processing data at certain hours of the day, it makes sense to host and then scale down to zero when it is not in use. Since it’s so small, we can also just containerize it and then host it on ECS or even Lambda for serverless inference. When using the closed sourced LLMs