About a month and over 100 commits later, it's almost done. I'm closing in on the final stage of the radiance interface rewrite. All modules compile without warnings and all that's left to do now is to test their functionality and fix all bugs that will inevitably arise during this. After that I can finally merge back into master and push the new radiance version.
There were a bunch of challenges along the way of this, but all in all it went by quite smoothly. Especially surprising was how little effort it ended up being to fix up the modules for the new system. The biggest hassle were the actual interface macro itself, which unveiled a couple of problems to me that I had not considered before, and the new module system based on ASDF.
To make my idea actually work I had to change a few things. First of all, any primary interface function has to be a macro. Inlined functions will not do as they do not have access to the &WHOLE feature of macro lambda lists. Second, the interface macro needs to expand into yet another macro before expanding into the final macros that we want. This is necessary because at the time of the primary macro-expansion, the package used for the interface has not yet been created. As such, we cannot INTERN symbols into it. The solution is to expand into a DEFPACKAGE and a MACROLET that is then executed pretty much immediately. The MACROLET itself then contains the INTERN/FIND-SYMBOL statements we want. It also took me a couple of tries until I got the lambda-list juggling right and trying to untangle and encapsulate the whole interface definition macro too took a bit more effort than I thought it would.
Aside from these changes out of necessity, I also added a few nice extensions to the original. For example, it is possible to define not only functions and macros, but also classes and since yesterday other “component types” as well. These component types can be user defined, so you can extend the interface definition for convenience types, such as the additional ACCESSOR, which aside from the method definition also automatically creates SETF and GETDF methods for it. This accessor in particular though unveils a bit of a problem that I cannot resolve: Interface functions are not SETF-able, as they expand into a FUNCALL or APPLY before the SETF does its magic. Therefore a SETF call to an accessor has to be done through either GETDF or the explicit interface method.
Next to the rewrite of the interfaces, the module system needed to be rewritten completely as well. Modules are now not their own classes or objects, but merely three different things that are linked together: An ASDF system of the module class, a package in which all module functionality resides, and a module identifier (usually a keyword). Since each system is an instance of the module class rather than its own class, and the instance found through ASDF may change at any time, it's not possible to use the system itself for interface dispatch. This in turn gave rise to the identifier, which is used as an EQL specializer on interface methods. The identifier of the current context can be retrieved through the package, which allows macros such as DEFINE-INTERFACE-METHOD and DEFPAGE to operate without an explicit identifier.
Thanks to the obliteration of module classes and the horrid DEFMODULE, a module can now only consist of two files, both of which are standard: an ASDF file with the system definition in it and an actual source file. This also streamlines the way information about a module is retrieved, as now all the known ASDF fields and functions can be used, rather than having it separate. Another bonus is that now modules can put interfaces into their DEPENDS-ON to assure that a certain interface will be present, without needing to explicitly state which module.
Making it all work with ASDF was the last big hurdle and it cost me quite a few of painful hours. Reading the ASDF source is not pleasant, as it's all one giant, 10k-lines file of condensed Lisp with about as few comments and docstrings as there is grass in the desert. Reading it is to me akin to trying to read an ancient grimoire of the elders while being a pansy magic apprentice. Whatever the case, after a few headaches and the realization of an incredibly stupid mistake of mine I managed to get it all rolling regardless. So, each interface definition now also expands into an ASDF system definition that delegates its loading to the system defined in the radiance configuration.
In the process of overhauling things I also changed the trigger/hook system. Hooks are now defined through a DEFINE-HOOK macro, which contains a body to call. This possibility arose as another consequence of the removal of module classes. Triggers can now also not accept arguments anymore, as I did not see the usefulness behind them as significant enough anymore.
One last rewrite aspect of the core that I will be tackling still is the refactoring of server commands into an interface. I've always wanted Radiance to be an abstraction layer that covers about everything, so making the server interaction an exception to that isn't following ideals properly. Since I've already got most of the functions encapsulated in Radiance this shouldn't be much of an issue.
When all is said and done, I will be able to move on to Ticker, my project and bug tracking system. Once that is done and running public, I will move on to the Kana project, which will probably halt development for a long while after its development completion. In general though, I will have to slow down, if not halt, everything in February as the new semester will start. This time I cannot allow myself to just wade through it like I did with the previous one; I really need to pour my attention into studying, or I am sure to fail horribly.
What I might be working on though is a documentation for Radiance, which should help me to revisit everything from a different viewpoint and also make it a more accessibly project to everyone else, as it'll give an easy overview of what it can do and how.
But that's all for later. Now I need to focus on Digital Circuits for a while and will continue my Radiance work after that, hopefully being able to round everything up before the start of the semester.
Written by shinmera