Hi,
I've a bit of a corner case and would like to ask for some perspective. First of all, the service I've developed and describe here is working fine, but for commercial / NDA reasons, I cannot share the code. I hope that is not too much of an issue although this really a is conceptual question. Can add sample code if needed, though.
Here is my situation: I've a third-party API from which I load reference data. Because at peak time, their servers are over-loaded and I am getting time-outs, I wrote an API proxy that downloads, filters, and processes all reference data outside peak hours and then provides the much reduced data-set to my system. Because the resulting data-set is only a tiny fraction of the original reference data, the first draft of the service just serves these data from memory. Also, these data are used everywhere, in development, testing, CI, production, etc. so co-locating the API proxy to my system solves a lot of problems. This is working fine for now.
However, because the reference data tend to change at a relatively slow pace and the third-party updates the full-data set without any notification or CDC, I have to diff the incoming reference data after download.
There are obviously many ways to go about this. One way is to diff and dump into a database and serve it from there. I can certainly do that. However, as the processed dataset is so small, we're talking about 50 MB meta-data, I am thinking along the line that, well, just set a chron schedule, re-download & process the data-set and then, well, yea.. And there is the crux, can you actually restart or reload a running warp service?
The way I understand the situation is that you can't, really. I might be wrong here, but because the serve function is the last one in main and is blocking main, I can't really see how to reload that one.
I am not married to any framework, I just used Tokio & warp because all other services are build with Tokio, but the bigger question really is, can you reload or restart any running web-service in Rust without re-deployment?
I did some search online but somehow people haven't really looked into this much. I did exacty this while ago in Golang for a very specific situation in which a microservice needed to be fully re-configured at run-time without re-deployment and it did worked out. Basically, I wrote the init process in such a way that you could re-run it programmatically and then called that function from an authenticated REST endpoint.
When I tried to port the code to Rust, I've bumped into the obstacle that I couldn't figure out how to reset or reload a running warp service let alone figure out if its even possible in Rust.
Any ideas, links, or leads?
Thank you in advance for any ideas or perspective.