Advice on build a test suite for a parser of the Rust language


#1

I’m writing a parser for the Rust language. And I’d like to have a test suite (basically a comprehensive set of examples of valid and invalid rust code).

A first thought is to reuse tests from rustc. The problem is, there a a lot of tests in src/tests, and I am only interested in tests for parser. There is a nice directory called tests/parse-fail, and what I would ideally like to have is tests/parse-pass.

To specify it more formally, I want to extract the smallest number of test files, which cover the most of libsyntax/parse/parser.rs. Any thoughts on how one may do this?

I guess I can run each test with coverage, and then solve a set cover problem, but it seems way too complicated: measuring coverage, running/building rust tests and solving an NP-hard problem are hard tasks even separately :smile:


#2

Well, the Readme in src/grammer/ uses src/**/*.rs as test suite for passing files… so, that doesn’t help in reducing the number of files at all :smile:

Thinking outside the boxrepo for a second: Have you looked at the tests of other rust tools, e.g. the rustfmt tests? They may not be very complete in regard to edge cases, but could be a good starting point as they try to cover every syntax variant rustfmt can format.


#3

I would love to have a documented, formal grammar, if that ends up coming out of this work :slight_smile:


#4

CTRS. This is (intended to be) a reusable conformance test for alternate implementations of Rust.

It’s not in a completely working state right now.

This is probably a better base to work from the the official tree, since I’ve already weeded out the ‘weird’ cases.


#5

It definitely will not be a formal grammar, just another parser generator thing like the one in src/grammar =( But I must say that libsyntax and src/grammar are quite usable as a source of grammar, especially because they cannot (sort of) become obsolete =) Though, a README in libsyntax, saying that the parser entry function is at the very bottom of the file, would be really helpful :smile:

Just perfect!