Must lexer tokens be surrounded by whitespace?

Discussion Forums discussion Must lexer tokens be surrounded by whitespace?

This topic contains 0 voices and has 1 reply.

Viewing 2 posts - 1 through 2 (of 2 total)
Author Posts
Author Posts
August 26, 2010 at 5:11 pm #5077

RichardRussell
Member

I’ve emailed Simon directly about this, but got no reply, so I’m trying again here. I’d like to write a PN lexer for BBC BASIC, but it has unusual requirements: keywords do not (necessarily) need to be surrounded by whitespace. For example the following is legal syntax in BBC BASIC (although not encouraged!):

x = DEGACSCOSPI

where ‘DEG’, ‘ACS’, ‘COS’ and ‘PI’ are all separate keywords! Can a custom PN lexer handle this situation, and if so where can I find out how to write it?

August 26, 2010 at 10:01 pm #17334

simon
Key Master

Hi,

Sorry, as you’ll note it’s easier to get a support reply on the forums, e-mails tend to get lost in the deluge!

In order to support this form of keyword combination you’d have to write a lexer using C/C++. The xml-based lexers all assume non-keyword characters as delimiters – it aims to cope with common requirements.

If you’re interested in working on a lexer like that I can probably help you with further guidance.

Simon.

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.