Parallel downloads/streams with priority

help
1

Hi all,

I am (re)-building a podcast app (see the re-architect branch). One of the features will be downloading more episodes while streaming another. It should be possible to set the priority of each background download. The user should not have playback pause because other episodes are being downloaded in the background. I have an idea on how to implement that but it feels rather hacky to me. Let me explain.

There would be an ordered list of futures

  • the first future in the list is the one streaming the data for the episode being played
  • the others are downloading episodes in the "background" the are in order from highest to lowest priority

When the future responsible for streaming the playing episode starts to lag we pause downloads starting with the lowest priority until we have enough bandwidth again.

Yet another approach would be making the futures take turn, returning to the first future when it needs data again.

Do any of you have suggestions on books or blogs where I can read up on the field of streaming? I would also like to know how to limit/throttle the overall bandwidth the downloads+stream cost. I could not find any myself but I might be missing the right search keywords.

And am I thinking along the right track here?

Thanks a lot for your time!
David

2

That sounds pretty good.

The only thing is, most podcast servers can serve a single episode in a couple seconds. The client won't benefit from downloading more than one at a time unless they have a faster connection than the server[1], and if that's the case, they don't need to worry about waiting a couple seconds for the first one to finish.

Parallel downloads could be useful if the server is slow, or if the client has a fast connection and the episodes are very large (i.e. video).

I'd guess for throttling, you'd need to get the network usage of your process from the OS and limit it that way. Easier but less accurate is counting bytes on the process side, but remember to do that before decompression, and you'll probably be missing some overhead from HTTP/TCP unless you have low-level access.

  1. ignoring startup/shutdown overhead ↩︎