BNF Grammar to Rust syntax

help
1

This question is for a rust programmer who more or less has built an interpreter, or has some experience working with BNF and Rust syntax.

I have done plenty of reading, but now I need some help pertaining to the implementation of a BNF Grammar into rust source code.

I have looked at the UNICODE standard. I have looked at ucd-generate.

How would a Rust programmer represent the following terminal symbols in Rust syntax:

simple Latin letter ::=
simple Latin upper case letter | simple Latin lower case letter

simple Latin upper case letter ::=
A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z

simple Latin lower case letter ::=
a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r | s | t | u | v | w | x | y | z

digit ::=
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

Thank you kindly.

2

Well, let's see... here's the moral equivalent to what I used in the last lexer I wrote.

pub fn is_simple_latin_letter(c: char) -> bool {
    is_simple_latin_upper_case_letter(c)
    || is_simple_latin_lower_case_letter(c)
}

pub fn is_simple_latin_upper_case_letter(c: char) -> bool {
    match c {
        'A' ..= 'Z' => true,
        _ => false
    }
}


pub fn is_simple_latin_lower_case_letter(c: char) -> bool {
    match c {
        'a' ..= 'z' => true,
        _ => false
    }
}

pub fn is_digit(c: char) -> bool {
    match c {
        '0' ..= '9' => true,
        _ => false,
    }
}

Nothing fancy. You could replace all the match expressions with the matches! macro... but I can never remember the syntax for that off the top of my head, so I tend to just use match.

Also, it depends on what you want these "terminal symbols" to actually do. For example, if you're asking how to actually extract something as opposed to just recognising it, that requires more effort (and a clearer explanation of what it is you want to do).

3

As a single expression, this is char::is_ascii_alphanumeric(), is_ascii_alphabetic, is_ascii_digit, etc.

4

Thank you kindly Daniel.

Im building an education platform. To do this Ive decided to build a SQL database so the users/teachers can make sql queries to the database, to find out more about their students. This would be part of the SQL language. Im not from a IT background so the terminology I use might not be consistent with the terminology you use, having said that, a lexer and parser for the SQL language. The database / SQL Implementation needs to be ISO compliant. Im working with the SQL language specification.

I will look into the macro. Is that answering your question?

5

Awesome, thank you kindly.

6

I mean this sincerely: good luck. That's a lot of work. :slight_smile:

Sort of. I was asking what you wanted to use the encoded BNF for. Those functions are, by themselves, only capable of testing to see if a single character matches the definition. That's not very difficult. But if you want to lex (for example) an SQL identifier, that's a much more complicated question that brings up questions of what approach you want to take to parsing.

There's numerous parsing libraries, and the answer to your question can change based on which one you're interesting in using.

(I don't use any of them. I write parsers as a recreational activity, so I hand roll all mine from scratch like a crazy person.)

3 Likes
7

It is a lot of work. Correct. Yes I'm crazy. Nothing ventured, nothing gained.

Ive built the front end -- which was easy for me. Not using rust. The front end is better than Oracle and Microsoft/ MySQL front ends - but it took me a year to build from scratch.

So now I need to build the BackEnd. It should take the whole of next year. Once that is done I will take the following year to build the API between the front end and the database. The glue. Finish off individual components.

Yes I would need it for lexing a SQL identifier. I have seen a sql parser online, in rust. I have two books which I'm reading. Dragon book, and Programming language pragmatics. Im struggling with separating the functionality of the database from the SQL language implementation. But once I have that -- it should be better sailing. :slight_smile:

Maybe one day I will bat in your league. Recreational activity.... :slight_smile:

8

But, but why not just use an existing SQL database? Building an ISO compliant SQL database sounds like many years of work, especially when starting from zero IT experience as you say. The desired educational platform may pushed so far into the future there is nobody who wants it when it's done. Hmmm.... unless learning to build an SQL database is what the educational platform is all about....

1 Like
9

I don't like using other peoples code / system of thought. SQL ISO Database is only one year. Getting started is the most difficult for me, but once I'm in the zone -- I make a lot of ground. Im hoping that once Ive built the database I will be able to transfer my ability to other domains of programming. Happy to take the risk. Im pretty confident that those people who don't have it, will be at a competitive disadvantage.

10

Hi, I know you mentioned that you are happy to spend a lot of time on this, and as a learning experience I think that is great. But just in case you are not aware, please know that it is not possible for a single person to successfully implement a modern and portable relational database implementation (and have it work for a usable length of time).

I just say this because you mentioned you don't have a background in IT, so I would hate for you to discover this only after having invested hundreds of hours into an attempt at the project. This is not only because of the sheer amount of effort required to write something like this, but also the cutting-edge research that goes into bringing these systems closer to optimality. To see this, you can take a look at the PostgreSQL repository, and at one of the seminal papers in relational database management systems versus a more modern one.

5 Likes
11

That is a fair comment, thank you kindly for the advice. Thank you for kindly for the links. I just need a basic system in the beginning and I will improve it over time. So I will keep it as basic as possible to get a working product.

12

You are severely underestimating the complexity of creating even a bare-bones database. Particularly if you're saying you have no background in IT, I'd say you have zero chance of succeeding in any reasonable timeframe. Seriously. "I don't like using other peoples code / system of thought." - so you're not even going to use existing Rust crates? That's called a "Not Invented Here syndrome", and it has a terrible reputation in IT, for very good reasons.

The most important things that a database should provide is ACID guarantees. That basically means "your data won't get corrupted". It's a critical property for any production-grade database, and it's really really hard to ensure. Many production-grade databases, with big experienced engineering teams, have gone for years without providing ACID. People using them got burned hard eventually.

That's even without touching a myriad of other big and small issues, like "proper support of SQL", or "good performance", or "query execution planning", or "scalability", or "consistency guarantees", or...

Seriously, get over it and use a proper database. You can start with SQLite if you want a simple file-based storage for your data. Or use some minimal PostgreSQL instance. Or even use a database managed by someone else, if you're deploying to a cloud provider. There are many options, depending on your specific use case, requirements, skills and affordable investments.

4 Likes
13

I would say it is valid when you start with what's available, see how it works and doesn't for what you're doing, so you have a good idea of what you need and don't need. This is not one of those cases.

I would suggest the OP should, if they wish to completely work from scratch, avoid targeting a relational database and instead go for append-only event streaming where you can always rederive the current state from the list of edits. This is much easier to implement and reason about optimizing and multi user situations, even if it doesn't make it easy (you still need to figure out what events to use and what they mean in conflicting situations, which is it's own branch of research)

1 Like
14

Thank you kindly for the Advice. When it's finished I will sell it to you. :slight_smile:

15

You may be interested in another Rust user's personal project to create a relational db from scratch.

16

It's ok, I have all the information I need. Thank you kindly.