Suitability for a working compiler

[The-Futurist]

I'm developing a systems programming language and have successfully developed the grammar with Antlr4 and a solid working front end AST builder and so on. All of this is written in C# and I'd intended to use LLVMSharp to leverage LLVM. This has progressed very well but I'm finding LLVMSharp to be a little hard to work with and I learned of LLVM.Net earlier today.

It looks more promising for a project like mine but I wanted to ask if there are any serious shortcomings or gaps in that would prevent a solid code generator for x64, ARM to be developed.

The language is not OO and is loosely based on PL/I but also leverages ideas from some other languages too. It's a compiled language that is still being designed but most of that is pretty much done with now.

Years ago I developed a PL/I compiler for Windows and that was written 100% in C long before stuff like Antlr or LLVM existed so I'm not what you'd call a novice, you can see that old project here if you're interested.

[smaillet]

I suppose, if I'm being honest, there is a potential drawback in that there's currently only one person maintaining all of this code base (ME), though I am interested in PRs/Collaboration. In the past it's gone without much activity, but I'm retired now, so I can "do whatever I want" 😁 And I'm still hooked on the idea of using this for the intended purpose of embedded devices.

No major shortcomings I can think of beyond the fact that it is currently limited to running on Windows Only. (Working on others, but it's actually a more complex issue than expected/hoped). It can TARGET other platforms but currently doesn't run on them. (Though I'd welcome help in that area!). MOST of it is pure IL so, not a serious concern, but the native wrapper is the real culprit there. The ultimate goal of all of this library is to support AOT generation of.NET IL for Embedded devices, so it's right in the same vein as a custom language. Though the Kaleidoscope tutorials illustrate it's use for a JIT/DSL.

There are some ANTLR generic utils in this repo and there is more from other work that isn't here, but the intent is to consolidate all of that into more generically useful context (Only the things unique to this repo should be here really). In particular we have a significant amount of support for using ANTLR grammars in VS as part of language providers etc....

The LLVM 20.1.X variant has been a focus for some time now as it was only on LLVM 10 for a long while. During the wait for LLVM to stabilize there have been a LOT of minor releases (They just this week released 20.1.8!). So, I was focused on getting the Versioning libraries working better/correctly. (CSemVer-CI has LOTs of subtleties and a not great spec). Now that those libraries are coming back together, I'll be publishing release of them shortly along with pre-releases of support for 20.1.8. I can then focus on anything that's new/missing in the topmost layer of Llvm.NET to expose more of LLVM.

[smaillet]

NOTE: I won't Trash on LLVMSharp as it isn't "bad" it's just "different". It operates at a conceptual equivalent of the "interop" layer for Llvm.NET. One of the main goals of the library is to HIDE all of that low level detail and establish a clean .NET wrapper that exposes things as close as possible to the underlying LLVM OO model while still remaining .NET. Another major goal was that the managed to native relied ONLY on the stable C-ABI. (While some folks have done the hard work to support a C++ ABI it has a LOT of limitations and is fragile since C++ itself does not guarantee or provide a stable ABI even between versions of the runtime with the same compiler and platform. (Lots of vendors jump through hoops to minimize the impacts of that but the language doesn't guarantee anything) Thus, the interop layer is strictly C based. So, there are differences in approaches dictated by the general needs and uses of the underlying LLVM.

[The-Futurist]

Thank you for taking the time to reply. OK I understand the goal, managed code on MCUs' I dabbled with .NetMF some years back (I studied electronics before settling into programming). More recently I did some projects on STM32 devices and though fascinating C got me down, I used C for years but always found it to be shallow.

I played around last year with GHI's Endpoint stuff (runs .Net) but soon found that was running LINUX and that was getting in the way of what should have been slick C# interaction with the device.

Fast forward and I've now restarted a project I toyed with about two years ago, a superior systems programming language for these devices.

So although I'm not focused on .Net I am certainly seeking to get this new language use to write code for these devices as opposed to C.

Here's the repo for this language.

Also here is some C# code I hacked on this morning using LLVMSharp to get some kind of crude LLVM generated. It's feeble but the generated bitcode does link and run a Windows exe.

https://github.com/Steadsoft/Syscode/blob/main/LLVMSandbox/Program.cs

Please take a look at how that LLVMSharp looks, I'm interested in how that same code would look if I used LLVM.Net

[smaillet]

Interesting project - I'd have to study the app to get a sense for what it does, to match it to Llvm.NET. But from a glance it looks like it's just creating a module and a single function. That's covered in more details with the sample CodeGenWithDebugInfo sample. The documentation for that covers more details. [The docs are from v10.0.0 as there has not yet been an update for LLVM 20 bits, but that is coming in a matter of days as a pre-release] The sample covers the basics of generating the equivalent IL to what Clang does for a given input C file. It demonstrates setting up the library, creating a context, module and function as well as dealing with types (structures in this case) and debug information. (So, it's a bit more than what your sample is doing but is further along the line) The biggest differences are:

• Support for multiple processor types and target ABIs (abstracted as an interface)
• Support for debug information
• Function attributes (to replicate what Clang would do)

[The-Futurist]

That crude app is just me exploring, I've never used LLVM at all, only read about the IR. So I was looking at how that kind of code might look once I design the code generator. Yes it's jut creating silly little functions and so on, but in the coming months I want to begin in earnest generating IR (bitcode) from basic syscode language fragments.

The language is a compiled imperative non-OO language but with solid systems support, flexible bit fields (bit is actually a data type), flexible control over padding and alignment of stuff, support for binary fractions and arbitrary binary sizes and more.

The compiler I built for PL/I in the 1990s has prepared me well for this too.

[The-Futurist]

@smaillet - I took a look at the sample you mentioned above and it looks quite well designed, the code looks as I'd "like" such code to look.

I'm reaching the point where I will be able to take source code and generate LLVM for it, something like this:

Generating a bitcode file for that and running clang on it will produce a Windows .EXE that will print the sum 42 in the console. (the printf is convenient because Clang can resolve it and link it, not intended to be part of language as such).

For that to work I need to create the AST validate it and then create the symbol table for the proc calc_demo so that the declared items are of known size and alignment and so on. Well that's already done (the error, red is just because the final form of namespace syntax isn't finalized):

(a symbol is a C# class used to describe low level details for the code gen, whereas a declare is an abstract representation of what was coded in the declaration, we create a symbol node from a declare node (and we create a declare node from a antlr declare context node)).

So I'm looking to do that in the near future, the compiler is in good shape and building solid AST and symbol tables, certainly enough to generate demo code for simpler cases (not structs, no arrays etc yet because although the AST has all that detail I haven't yet included it i the symbol table.)

[smaillet]

FYI: 20.1.8-alpha was released today.

Once you get your head around the sample codegen app, it's worth looking into the Kaleidoscope sample as it covers a functional language implementation. Most of the tutorial is focused on code gen with a JIT, but chapters 8 and 9 are focused on and provide a full source AOT compiler (Source -> Object file).