How to parse real numbers?

[duskmoon314]

Hi. I am trying to implement a parser of positive real numbers. The format is expressed in regular expressions as follows:

([1-9][0-9]*|0)(.[0-9]+([eE]-?[0-9]+))?

The valid value would be
0
123
123.456
0.1
0.2E5
0.3e-7

My attempt:

pub fn real_number() -> impl Parser<char, Token, Error = Simple<char>> {
    text::int(10)
        .chain::<char, _, _>(
            just('.').chain(
                text::digits(10)
                    .chain(
                        one_of("eE")
                            .chain(just('-'))
                            .or_not()
                            .flatten()
                            .chain(text::digits(10)),
                    )
                    .or_not()
                    .flatten(),
            ),
        )
        .or_not()
        .flatten()
        .collect::<String>()
        .map(Token::RealNumber) // enum Token{ RealNumber(String) }
}

I'm still unfamiliar with chain, or_not, and flatten. So the above code can be compiled but cannot parse as I want:

// get "123."
real_number().parse("123.456")

// get "0."
real_number().parse("0.1E5")

I wonder how to correctly build the parser.

[zesterer]

I think you're fairly close with this as-is. You might find the number parser in the json.rs example to be useful, it pretty much does what you're looking for.

Lexing tokens like this is definitely the sort of thing that's currently a bit verbose in chumsky today. This will be improving though! The soon-to-be-merged zero-copy branch will include a regex combinator, allowing you to parse something like this in a single line!

[duskmoon314]

I just tried the example in json.rs, which is quite helpful! But I am confused that the parser doesn't deny some wrong patterns.

// Modified from json.rs
pub fn real_number() -> impl Parser<char, Token, Error = Simple<char>> {
    let frac = just('.').chain(text::digits(10));
    let exp = just('e')
        .or(just('E'))
        .chain(just('-').or_not())
        .chain::<char, _, _>(text::digits(10));

    let real = text::int(10)
        .chain::<char, _, _>(frac.or_not().flatten())
        .chain::<char, _, _>(exp.or_not().flatten())
        .collect::<String>()
        .map(Token::RealNumber);

    real
}

// expect to get an error
assert!(real_number().parse("0.1E").is_err());
// get:
// thread 'main' panicked at 'assertion failed: real_number().parse(\"0.1E\").is_err()',

[zesterer]

It might be worth reading the tutorial page. The key piece of information you're missing here is that parsers are lazy by default. This means that they'll only parse as much as they can correctly and no more. If you want to force a parser to consume all input, you need to add .then_ignore(end()). If you add this after the real_number() call, you should get the expected results.