What is best way for Distributing?

#1

We work on a project that need some nodes communicate with togheter to much with :

  • high performance
  • non-blocking
  • backpressure

and a fast serialization
( i think binary format is fastest )

I think tonic grpc is best for this !!

do you have better idea ?!

#2

Something like gRPC is reasonable. Another option is to simply open a TCP stream and send messages in the format provided by bincode.

1 Like
#3

Thanks. If you do something like apache spark for dispatching task to nodes .

Which do you choice ?!

#4

And one of thing i dont like in raw tcp it is, when we don't know length of object , and need buffer and sometimes may need to buffer data and wait for other data to all be available.

#5

If you use bincode, then you would generally write the length of the next message before the message. Any RPC library will do something similar internally.

I'm not familiar with apache spark.

1 Like
#6

Hi,

Some other suggestion, ZeroMQ for data transfer and Apache Avro for serialization/deserialization.

1 Like
#7

hi. thanks

#8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.