I would like to write a small program that prints out information which it receives from the network (ideally using asynchronous I/O) while at the same time allows the user to input data in the same terminal.
Does anyone know a good set of libraries/crates and/or typical approaches how to do this by yourself?
You can use crossterm::event::EventStream to get terminal input in an async environment. crossterm also has all the other fundamental terminal input/output facilities you might want.
Note that if you want the user's input to be cleanly separated from the output, you need to provide your own line editor/buffer to redraw, to keep the output from being mixed up with individual keystrokes. There are a few Rust libraries offering that, but I haven't tried them myself yet.
(tui, which can use crossterm as its backend, is neither of the above; it only deals with visual layout of "widgets" and efficiently sending the results of that layout to the terminal. It's useful if you want more of a full screen UI layout than "one input line and stuff scrolling up above it".)
Thank you for referring me to tui and crossterm. I played a bit with it, but didn't get so far (yet).
use crossterm::event::{Event, EventStream, KeyCode};
use crossterm::execute;
use crossterm::terminal::{
disable_raw_mode, enable_raw_mode, EnterAlternateScreen, LeaveAlternateScreen,
};
use futures::StreamExt;
use std::io;
use tokio;
use tui::widgets::{Block, Borders};
use tui::{backend::CrosstermBackend, Terminal};
#[tokio::main]
async fn main() -> Result<(), io::Error> {
enable_raw_mode()?;
let mut stdout = io::stdout();
execute!(stdout, EnterAlternateScreen)?;
let backend = CrosstermBackend::new(stdout);
let mut terminal = Terminal::new(backend)?;
let mut reader = EventStream::new();
terminal.draw(|f| {
let size = f.size();
let block = Block::default().title("Block").borders(Borders::ALL);
f.render_widget(block, size);
})?;
loop {
match reader.next().await {
None => break,
Some(event) => {
let event = event?;
if event == Event::Key(KeyCode::Esc.into()) {
break;
}
}
}
}
disable_raw_mode()?;
execute!(terminal.backend_mut(), LeaveAlternateScreen)?;
terminal.show_cursor()?;
Ok(())
A lot of code which basically just draws a box and waits for the escape key to be pressed. It's mostly a mixture the example code of these crates and @kpreid's hint to use EventStream. It doesn't refresh the output yet, etc.
I overall feel like it's a lot work to do for the simple task of having a (scrolling) output "window" and a text input.
To me, things like enable_raw_mode, EnterAlternateScreen, terminal.show_cursor etc. feel a bit too low-level for what I want. But perhaps it's not much more than these few things I have to care for, but not sure. I feel like the approach keeps reminding me of "contolling a terminal" even when I want to program on a higher-level.
Maybe I should instead make a simple graphical application? But also in that case I'm pretty unsure which crates to use. I know I started this thread on text-mode applications, but perhaps someone knows a good way to use a GUI for such a "text" interface (basically would be a text area plus an input box, I guess). But I'm also happy for some more hints on the terminal approach.
Perhaps I should just work through this, but I was hoping for something more simple to use and more easy to read. I guess I need to figure out how to use the widgets of tui, and also not sure yet how and when I have to redraw. I would assume I have to "catch" resizing events and then manually trigger a redraw?
I'm sure I can figure these things out with time, but if someone has already done it, a few tips (or a link to the right example) might help me to understand better the overall workflow of such an application using tui and tokio & crossterm::event::EventStream.
I overall feel like it's a lot work to do for the simple task of having a (scrolling) output "window" and a text input. … Maybe I should instead make a simple graphical application?
A graphical application will likely have even more boilerplate to do anything interesting, though the exact amount of code will vary wildly by which 'framework' you use. The only place I've seen an input/output UI be even more compact is in HTML+JavaScript, or other “always GUI” programming environments.
On the other hand, in almost any GUI you'll get more functionality for free, like proper input editing, a scrolling text box, etc.
For the sake of demystifying the problem, I took your code and expanded it into a toy version of bidirectional communication with an async task, with a minimal line editor (backspace only). I also simplified some of the code you already had — we can create the Terminal at the start and do everything else through its backend.
Disclaimer: I haven't actually seriously worked with tokio's concurrency tools before and there may be better ways to do things (this program definitely won't behave right in some error circumstances, such as the spawned task exiting its loop):
use crossterm::event::{Event, EventStream, KeyCode, KeyEvent};
use crossterm::execute;
use crossterm::terminal::{
disable_raw_mode, enable_raw_mode, EnterAlternateScreen, LeaveAlternateScreen,
};
use futures::StreamExt;
use std::io::{self, Write};
use std::time::{Duration, Instant};
use tui::layout::{Constraint, Direction, Layout, Rect};
use tui::widgets::{Block, Borders, Paragraph};
use tui::{backend::CrosstermBackend, Terminal};
#[tokio::main]
async fn main() -> Result<(), io::Error> {
enable_raw_mode()?;
let mut terminal = Terminal::new(CrosstermBackend::new(io::stdout()))?;
execute!(terminal.backend_mut(), EnterAlternateScreen)?;
// Async task that emits log lines periodically and can be controlled:
let (input_s, mut input_r) = tokio::sync::mpsc::channel::<Option<String>>(100);
let (output_s, mut output_r) = tokio::sync::mpsc::channel::<String>(100);
tokio::spawn(async move {
let mut state = Some(String::from("Hello world"));
loop {
tokio::time::sleep(Duration::from_secs(1)).await;
if let Ok(new_state) = input_r.try_recv() {
state = new_state;
}
match state.clone() {
Some(string) => {
output_s
.send(format!("{:?} {}", Instant::now(), string))
.await
.unwrap();
}
None => {
break;
}
}
}
});
// UI state
let mut log: Vec<String> = Vec::new();
let mut input_buffer = String::new();
let mut reader = EventStream::new();
loop {
terminal.draw(|f| {
let [output_size, input_size]: [Rect; 2] = Layout::default()
.direction(Direction::Vertical)
.margin(1)
.constraints([Constraint::Min(3), Constraint::Length(3)])
.split(f.size())
.try_into()
.unwrap();
let output = Paragraph::new(log.join("\n"))
.block(Block::default().title("Output").borders(Borders::ALL));
let input = Paragraph::new(&*input_buffer)
.block(Block::default().title("Input").borders(Borders::ALL));
f.render_widget(output, output_size);
f.render_widget(input, input_size);
})?;
tokio::select! {
log_line = output_r.recv() => {
// TODO: if the sender is dropped we should stop checking this future
log.push(log_line.unwrap());
}
event_result = reader.next() => {
let event = match event_result {
None => break,
Some(Err(_)) => break, // IO error on stdin
Some(Ok(event)) => event,
};
match event {
// Quit
Event::Key(KeyEvent {
code: KeyCode::Esc, ..
}) => {
break;
}
// Delete character
Event::Key(KeyEvent {
code: KeyCode::Backspace, ..
}) => {
input_buffer.pop();
}
// Send line
Event::Key(KeyEvent {
code: KeyCode::Enter, ..
}) => {
input_s.send(Some(input_buffer.clone())).await.unwrap();
input_buffer.clear();
}
// Type character
Event::Key(KeyEvent {
code: KeyCode::Char(c), ..
}) => {
input_buffer.push(c);
}
_ => {
// Unrecognized input
write!(terminal.backend_mut().by_ref(), "\x07")?;
terminal.backend_mut().flush()?;
}
}
}
}
}
disable_raw_mode()?;
execute!(terminal.backend_mut(), LeaveAlternateScreen)?;
terminal.show_cursor()?;
Ok(())
}
I have been using HTML+JavaScript before. I agree it's easy to quickly get something running, but doing things right (session management, CSS, asynchronous requests in the background, etc.) can get things quite complicated also. Maybe anything involving UI will get complex quickly when you do it right.
In my current case, I only have the need for text (but want it to work asynchronous, so I thought I could end up with something more simple.
The use-case is actually a tiny chat program for amateur-radio that I was considering to write. But being able to write small and lightweight text apps might come in handy for other things as well.
I feel like I should perhaps go that way, even if – for now – I only need text. If I understand it right, then using something like GTK will "trap" me in the main event loop? So this wouldn't work with tokio directly? I guess for my I/O I can start a different thread which then can use async I/O using tokio and will send events through GIO, DBus, etc.? Hmmm… I just found this, so maybe this will help me: GUI development with Rust and GTK 4.
I think GTK is the most famous choice for GUI programming with Rust? At least that was my impression when searching for resources on it.
Thanks for the example. It looks nice (and not too complex yet). I feel like things will get more complex when making a real application though, because of:
Btw, what I found interesting is that enable_raw_mode will keep me even from SIGSTOPping the process with CTRL+Z. It's unusual to have programs exhibiting such behavior (a lot of programs will ignore CTRL+C but still allow CTRL+Z to pause the program so you can kill it for example).
When I went through crossterm's examples, I came across a call of FutureExt::fuse that I did not understand: examples/event-stream-tokio.rs
let mut delay = Delay::new(Duration::from_millis(1_000)).fuse();
let mut event = reader.next().fuse();
select! {
_ = delay => { println!(".\r"); },
maybe_event = event => {
/* … */
}
};
I did not understand why fuse is needed here. Do these futures behave odd if they are polled after they have been Ready? Does tokio::select! try to poll futures that have been previously Ready? I'm not familiar with async Rust enough to really understand what's happening here.
Whichever way I go with my small program, I feel like it's not going to be a simple task, even if all I want is text input and output.
If I didn't have events while waiting for input, I simply could use rustyline. But I think reading a line is always blocking with rustyline as I have to call rustyline::Editor::readline. I would like if I could just interrupt text input when there is a new message to be displayed, output the text (using normal terminal scrolling capabilities), and then restore the current input line buffer on the last line in the terminal. But I don't think I can do that with rustyline.
Sorry to mix up all sort of different topics in this thread and post, let me try to summarize some of the open questions I have (which perhaps someone knows answers to):
What's the best choice right now to make a GUI? (I know this question could likely trigger another thread, and perhaps that's what I should do. Or look through one of the previous threads:
But maybe there is a quick answer to questions like gtk4 has better support for Rust than QT or vice versa. (I have worked with none of these yet.)
I'd like my application to run on several platforms (Linux, Windows, macOS, FreeBSD, …).
Are there some simple terminal line-editors that can work async and which support being interrupted by other events? I.e. something less powerful than tui regarding output, but comparably powerful to rustyline regarding input (and working async)?
Does someone know while that example above needs .fuse()?
If I understand right, you simply redraw the whole screen on each event (including each keystroke). That seems to also ensure that when I resize the window, the boxes get all redrawn.
I would assume tui has some sort of internal buffer, so the content isn't really re-printed on every keystroke? (As this might be bad on low-bandwidth remote connections.)
If I understand it right, then using something like
GTKwill "trap" me in the main event loop? So this wouldn't work withtokiodirectly? I guess for my I/O I can start a different thread which then can use async I/O usingtokioand will send events through GIO, DBus, etc.?
Every GUI framework that works this way (which many do) will have to include some mechanism to queue something to be run in the event loop, from another thread. So, Tokio has its own thread pool, and when the UI needs updating, your async task schedules a callback (or some kind of “custom event”, depending on how the framework approaches this) to call your code running on the UI thread and have it update the UI.
Btw, what I found interesting is that
enable_raw_modewill keep me even fromSIGSTOPping the process with CTRL+Z.
This is (on Unix, at least) because raw mode disables the interpretation of ^C, ^D, ^Z, etc. which is normally supplied by the kernel tty/pty driver (which is also what does character echo and line buffering, the things we need to turn off). Because of that, it's IMO a good idea for your program to explicitly recognize those conventional keystrokes.
But I think reading a line is always blocking with
rustyline…
If I were trying to write this myself, I'd try noline (which I have not used, only read the announcement of):
The key feature is that it provides an editor algorithm which does not do IO itself (nor expect a callback that performs IO), so it can be adapted to any environment, including your async one. (This design principle is sometimes called "sans-I/O".)
If I understand right, you simply redraw the whole screen on each event (including each keystroke). That seems to also ensure that when I resize the window, the boxes get all redrawn.
Yes. This is a GUI API style known as "immediate mode": the framework does not track what objects exist, but requires you to specify them anew every frame. It is of course less efficient for highly complex UIs, but it saves a lot of change-propagation logic (both in the framework and sometimes for you).
I would assume
tuihas some sort of internal buffer, so the content isn't really re-printed on every keystroke? (As this might be bad on low-bandwidth remote connections.)
Yes. This buffer is actually much more important than just for efficiency: it also ensures that the UI will never flicker if e.g. a dialog is (re)drawn on top of other content, by buffering all the drawing and only at the end figuring out what should be sent to the real terminal.
This is also a very traditional thing to do for terminal UIs, dating back to the days of applications connected over serial or phone lines with what we would consider severe bandwidth limitations. The curses C library (dating back to 1980) does exactly the same buffering technique.
I tried to put something together. But it doesn't fully work as I would like. Let me share the code:
[dependencies]
tokio = { version = "1", features = ["full"] }
termion = "1.5"
noline = { version = "0.2", features = ["std", "tokio"] }
use termion::raw::IntoRawMode;
use termion::clear;
use noline::builder::EditorBuilder;
use tokio::io::AsyncWriteExt;
use std::time::Duration;
#[tokio::main]
async fn main() {
let raw_mode = std::io::stdout().into_raw_mode().unwrap();
let mut stdin = tokio::io::stdin();
let mut stdout = tokio::io::stdout();
const PROMPT: &'static str = "> ";
let mut editor = EditorBuilder::new_unbounded()
.with_unbounded_history()
.build_async_tokio(&mut stdin, &mut stdout)
.await
.unwrap();
let (timer_s, mut timer_r) = tokio::sync::mpsc::channel::<()>(100);
tokio::spawn(async move {
loop {
tokio::time::sleep(Duration::from_secs(5)).await;
timer_s.send(()).await.unwrap();
}
});
loop {
tokio::select! {
input = editor.readline(PROMPT, &mut stdin, &mut stdout) => {
if let Ok(line) = input {
stdout
.write_all(format!("Read: '{}'\n\r", line).as_bytes())
.await
.unwrap();
} else {
break;
}
},
event = timer_r.recv() => {
if let Some(()) = event {
stdout.write_all(clear::CurrentLine.as_ref()).await.unwrap();
stdout.write_all("\rEvent occurred!\n\r".as_ref()).await.unwrap();
} else {
break;
}
},
};
}
drop(raw_mode);
}
Not sure if I used the interfaces properly, but the program kinda works. When an event is triggered, the current line gets erased and a message is printed:
% cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.01s
Running `target/debug/mycrate`
> Hi there!
Read: 'Hi there!'
Event occurred!
>
Interestingly, the prompt is even automatically restored afterwards (but without the currently typed line), and I don't know which component is doing that. I thought I have to somehow restore the prompt manually, but apparently this is done through some control sequence perhaps?
Anyway, this doesn't restore the line I was currently typing. At least it apparently clears the buffer, so when I type in something that gets interrupted, and then type something again, only the displayed text (i.e. the text that I type in after the event interrupted me) will actually be returned… 
… magic! 
I didn't see any way to access the current buffer in noline::no_sync::tokio::Editor. Besides, the returned future of Editor::readline will have a mutable borrow on the editor, so no way to access the contents here. I feel like this approach isn't really getting me where I want to.
But perhaps I could try to understand how noline works and implement something similar on my own, which allows me to peek into the current buffer.
Thanks for all the input and inspiration!
I don't have any personal experience to add, but this article walking through making an async tui project looks decent, inspired by spotify-tui.
I don't know if it's any good, but I was going through some bookmarks and happened to find tui_input, which is already designed to work inside of tui. Maybe that would be a better fit than noline. It doesn't seem to have had any development since the original release but it at least could be a working example.
Thanks a lot for the article, it may help me to get accustomed with tui.
Yeah, it looks interesting too. Here is an example. (I had to use crossterm version 0.22 instead of 0.23 when using tui_input.)
I assume I could replace the crossterm::event::read call here in the example with crossterm::event::EventStream, as you suggested in your first post in this thread.
It might take me a while to put all this together, but I certainly have a good starting point. Maybe tui (with or without tui_input) is the way to go.
I am not sure that it matches your needs but an external printer has been introduced in rustyline (currently available only in master branch): see here rustyline/external_print.rs at master · kkawakam/rustyline · GitHub
Amazing! That's exactly what I was looking for.
I tested it, and it looks promising. There seems to be a bug with my terminal though, (edit: it's not a bug, I was just accidentally using an older version, sorry for the noise!) because sometimes some lines get lost (it helps to reduce the timing in the example code so the error occurs faster):
% cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.02s
Running `target/debug/readnwrite`
External message #0
External message #1
External message #2
External message #3
External message #4
External message #5
External message #6
External message #7
External message #9
> This is exactly what I need, but some messages appear to get lost.
Line: This is exactly what I need, but some messages appear to get lost.
External message #10
>
See there is message #8 missing. 
But probably that's something that can be fixed. Maybe I'll later want some fancy (T)UI, so I would have to move to something like tui or Qt, GTK, etc., but I like the easy interface of rustyline when I just want to do something simple and quick!
It's not async, but I can work with threads too (I think), so not a huge problem.
Many many thanks for pointing me to this. I'll see to write a bug report on the missing lines (unless there is already one filed).
P.S.: I opened an issue here, where I described the problem a bit better. Apparently the terminal and/or program sometimes ends up in a bad state where no more messages are printed until I type in some more text (but the messages that weren't displayed are lost then).
P.P.S.: It's not a bug, I was just accidentally using an older version, sorry for the noise!
Note that there is now a rustyline-async crate for this. It is not related to the "real" rustyline and may be early in development phase, but can nonetheless be still of interest.
plus there's also reedline that i discovered in the process. it looks like it have nice basic feature set - though i don't know yet whether it supports async
(i am in phase of testing/checking for presence of all "nice to have features", async being lower priority for me currently, but definitely very nice to have for future)
Oh this seems to have the same feature I need (but with async), so may be worth looking into also.
Generally, I'd be happy if I can make the application async and avoid using threads.
Not sure if the rustyline authors have been asked, but I think it's a courtesy to not use other (existing) projects names deliberately for one's own project – at least not when they are so distinctive as "rustyline" is. This can cause confusion to users (apart from legal issues, depending on jurisdiction). But that's just my personal p.o.v. and may be seen differently.
I just tried to put together a small example with rustyline-async, and this is how it looks like:
[dependencies]
tokio = { version = "1", features = ["full"] }
futures-util = "0.3.21"
rustyline-async = "0.2.2"
use futures_util::io::AsyncWriteExt;
use rustyline_async::Readline;
use tokio::{select, time::sleep};
use std::error::Error;
use std::time::Duration;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let (mut rl, mut wr) = Readline::new("> ".to_string())?;
let mut wr2 = wr.clone();
let event_generator = async move {
let mut counter = 1;
loop {
wr2.write_all(format!("Event #{counter}\n").as_bytes()).await?;
counter += 1;
sleep(Duration::from_millis(1000)).await;
}
#[allow(unreachable_code)]
Ok::<(), Box<dyn Error>>(())
};
let input_loop = async move {
loop {
let line = rl.readline().await?;
wr.write_all(format!("Got line: {line}\n").as_bytes()).await?;
}
#[allow(unreachable_code)]
Ok::<(), Box<dyn Error>>(())
};
select! {
Err(err) = event_generator => { Err(err)? },
Err(err) = input_loop => { Err(err)? },
}
Ok(())
}
It's a bit ugly due to the Ok::<(), Box<dyn Error>>(()) hack to infer the type for the ? operator in the async block. But other than that, it looks pretty much simple. This might solve my problem.
But I'm happy to look at the original rustyline as well and/or other alternatives.
What I don't understand is how futures_util::io::AsyncWriteExt differs from tokio::io::AsyncWriteExt. I tried:
-use futures_util::io::AsyncWriteExt;
+use tokio::io::AsyncWriteExt;
And I get:
error[E0599]: the method `write_all` exists for struct `SharedWriter`, but its trait bounds were not satisfied
--> src/main.rs:15:17
|
15 | wr2.write_all(format!("Event #{counter}\n").as_bytes())
| ^^^^^^^^^ method cannot be called on `SharedWriter` due to unsatisfied trait bounds
|
::: /home/jbe/.cargo/registry/src/github.com-1ecc6299db9ec823/rustyline-async-0.2.2/src/lib.rs:326:1
|
326 | pub struct SharedWriter {
| -----------------------
| |
| doesn't satisfy `SharedWriter: AsyncWriteExt`
| doesn't satisfy `SharedWriter: AsyncWrite`
|
= note: the following trait bounds were not satisfied:
`SharedWriter: AsyncWrite`
which is required by `SharedWriter: AsyncWriteExt`
= help: items from traits can only be used if the trait is in scope
help: the following traits are implemented but not in scope; perhaps add a `use` for one of them:
|
1 | use futures_util::io::AsyncWriteExt;
|
1 | use std::io::Write;
|
1 | use tokio::io::AsyncWriteExt;
|
[…]
So apparently both tokio and futures_util have their own traits for AsyncRead / AsyncWrite. But I assume I can still use futures_util::io::AsyncWriter with the tokio::runtime::Runtime? I think that's because each Future, when polled, receives a Context with a Waker that the future's poll method can call as soon as more bytes can be written. But isn't there some sort of "main loop" that does the select/poll/epoll on all file handles that wait for I/O? I always thought I can't mix the different async engines.
There are extension traits for converting between the AsyncRead and AsyncWrite traits of futures and tokio; there is no fundamental issue with runtimes, only with backward compatibility.
And which would be the canonical trait to use for my own data types? futures_io::AsyncWrite or tokio::io::AsyncWrite? Or does it depend… and if yes, on what? Or will one of these traits be superseeded/replaced by the other in future?
Sorry for my dumb questions, I'm just not so familiar with async Rust yet. It's still pretty confusing to me.
Nah this is a common confusing mess. I've seen both types in use in the wild.
I think the goal is this gets added to std at some point (like Future) and we have three types!