New week new Rust! What are you folks up to?
Playing around with async support in elastic_reqwest and am hoping to jump on some of the awesome new libz blitz crate evaluations that have popped up this week.
I'm also teetering on submitting some talks to the RustFest Call For Proposals. It all depends on whether I can be sure of getting there. Living in Australia makes it hard to get around 
2 Likes
Moving forward with my experiment on high-rate /proc sampling. After my /proc/uptime proof of concept from last week, I interfaced a first "useful" file this week: /proc/stat.
As I could perhaps have expected, the performance is much worse. It takes ~19µs just to get the data out of the Linux kernel, and my initial sampler took a total of ~43µs (syscall+24µs) to do this and then parse and record the data. By optimizing the handling of whitespace (no need for the sophisticated Unicode-aware std::str::SplitWhitespace when parsing procfs pseudofiles) and unused IRQs, I took it down to ~29µs (syscall+10µs), and then the law of diminishing returns started to hit pretty hard.
And so...
- The more I look at it, the more I am amazed at how much of a terrible idea making OS kernel interfaces masquerade as text files is, as far as performance is concerned
- Shoving too many unrelated statistics in a single "file" is a terrible idea as well. Of course, too fine a granularity would be problematic as well (due to syscall overhead), but mixing "software" CPU activity and serviced IRQs really hurts more than it helps.
- On useful, reasonably complex procfs files, 1 kHz sampling at 1% overhead is unrealistic. 30~100 Hz sampling at 1% would be more realistic.
- For /proc/stat at least, it does not matter: the time resolution of the CPU statistics is too poor for faster sampling to be worthwhile.
2 Likes
I've been thinking about making a simple compression tool to help me as a beginner get my hands dirty with the syntax and borrow rules and such. (and because I've got really interested in string algorithms recently!)
Working on rust-tuf and to get key delegations finished. And hopefully writing an ungodly amount of much needed integration tests because the TUF spec is complex.
3 Likes
As you can see from the Wiki I started with hardcoded test scenes (or a single executable, which could render variations of the same scene). The last couple of weeks I wrote a parser using the pest crate, so the same scene can be parsed and rendered now from a scene description file. I just released (see release notes) another version of my rs_pbrt library, which can parse and render another provided test scene, but it's still missing the contribution of an area light source. So I guess I will work on that for the rest of the week.
What's interesting about the teapot scene is that is has 2328 triangles and the hardcoded version resulted in approx. 120k lines of Rust code (using my library) and took several minutes just to compile. The parsed version of the same scene parses and renders the same scene in less than half a second.
Another interesting fact mentioned in the release notes is how I avoided a fatal runtime error, a stack overflow, by spawning a thread with a bigger stack.
Warning: If you want to compile the pest_test executable you need to download and compile with Rust nightly.
2 Likes
I am not sure what you need exactly, but systemtap might be a better tool for that.
1 Like
Kernel instrumentation tools like systemtap are certainly incredibly powerful, but their great power comes with equally great responsibilities:
- Injecting arbitrary code in a live OS kernel has major implications on system reliability and security, which I would rather avoid thinking about when writing code.
- Because instrumenting the kernel is a major breach of OS isolation, it will always be a highly privileged activity. But I prefer to build tools which are available to unprivileged users.
- Kernel instrumentation uses source-level kernel interfaces, which in the Linux world are unstable. This is fine for throwaway scripts, but not for long-lived tools. I don't have the resources it takes to pull an NVidia and chase the perpetually moving target of Linux internals.
For these reasons, I would rather stick with user-space interfaces whenever they do the job.
BPF solves this problem. I didn't mean systemtap specifically, but all related kernel interfaces (it just easier to refere to them by one name, even if it is not precise).
Because instrumenting the kernel is a major breach of OS isolation, it will always be a highly privileged activity.
There is no breach of isolation really, but it does require root access now.
Kernel instrumentation uses source-level kernel interfaces
You don't need source for tracing. There are many things that are "stable enough". Sure, they may change, but for fundamental functions it is rather unlikely.
1 Like
Adding support for Mysql-specific data types to Diesel.
1 Like
I'm getting started with rust Looking to port my https://github.com/pwarren/rtl-entropy code I think I have a good idea on how to start, but we'll see!
1 Like
The contribution of the area light source is now working correctly (see release notes):
https://github.com/wahn/rs_pbrt/wiki/Release-Notes
Done for this week ... off to Amsterdam 
I'm still learning Rust, and trying to find and study methodologies. I designed a GUI toolkit, which is based on modular inheritance, so I'm experimenting with how that might work in Rust using traits and macros. Right now, stuck on dynamic dispatch + delegation methodologies. Very fun and exciting! 
I've finished the initial release of Digits. It's a character sequencer with built in mathematics allowing you to achieve perfect precision in multiplication of unsigned digits of any numeric base or character set choice.
Here's an example of base 10 multiplying the value of u64 MAX to the power of two:
let base10 = BaseCustom::<char>::new("0123456789".chars().collect());
let mut num_u64_max = Digits::new(&base10, "18446744073709551615".to_string());
let two = num_u64_max.propagate("2".to_string());
println!("{}", num_u64_max.pow(two).to_string());
// "340282366920938463426481119284349108225"
This wasn't designed for math specifically… it was designed for brute-forcing given sequences of characters by what ever additive step you'd like to take.
All this implemented via the linked list struct Digits.
3 Likes