I'm attempting to write something which limits total up/down bandwidth consumption over a specific time window. I need a way to calculate a rolling "budget" of sorts and I'm not sure how to do that but more specifically I need an http client where I can hook deep into the actual read/write calls to limit them and fit them into this budget.
For reqwest I saw that there is a way to hook into connection establishment but I didn't see a way to hook into the actual reads and writes.
Are there any http client libraries out there which support doing something like this?
Bonus points that might merit another question: how can I keep track of this budget in a data structure so that I can keep within a sliding quota over time?
My recommendation: This can be written as a sans-io library that produces serialized HTTP messages with the http and url crates and does its running budget accounting based on those. Most HTTP client crates worth their salt accept the http and url types as is. In other words, you can control send/receive limits with every HTTP client that directly supports the common interfaces.
It's easier said than done, but this is how I would do it. The library could be used for clients and also slotted in as middleware for servers.
Have a look at pingora-limits. It isn't the only one, but it's a sans-io library that does this kind of bookkeeping with various strategies.
Hey, since you want to limit the "bandwidth" consumption and you want to keep track of it in a structure its better to do it yourself instead of frankeisteining a couple of crates that themselves will possibly make your program less performant its better if you code it yourself, you should start with wrapping read and write and calculate their 'bandwidth' during a period of time,
just curious here, i'm wondering what you are are making and why do you need to limit the bandwidth? is the requested http server on the cloud or are you providing a paid service that depends on that budget for each client?