FreeM History

Since 2014, I have been the maintainer of the primary fork of the FreeM implementation of the M programming language and database. I thought I would take some time to share some of the history of FreeM, as well as its current status and goals.

How I got involved in M and FreeM

My mentor in computer programming and UNIX was Larry Landis, who got involved heavily in the M/MUMPS programming language ca. 1991. He hyped up the M language to me from 1991 forward, and first demonstrated FreeM to me in August 1998. In 2010, I incorporated my company, Coherent Logic Development, learned M, and began doing contract work in M through Larry’s company, Fourth Watch Software.

Larry was the owner of FreeM’s SourceForge repository, which had not been touched in a number of years, following Fidelity National Information Services’ decision to release GT.M under a free software license. In August 2011, I downloaded the source code for FreeM and did enough work on it to get it running under modern GNU/Linux systems and posted it to the mumpster.org forums.

In 2014, Larry gave me administrator access to the FreeM SourceForge repository and transferred maintainership of the project to me.

Early History of FreeM

FreeM was developed in Germany in the mid-1990s by a developer who went by the pseudonym “Shalom ha-Ashkenaz”, whose actual identity remains unknown, though it is thought by some that he is or was a dentist who learned C and developed FreeM on his own time. Shalom developed FreeM at a time when Terry Ragon of InterSystems (the company that developed the ISM implementation of M) was buying up all of his competitors and shutting them down. Shalom wished to provide a community-driven, open-source implementation of M as a bulwark against the growing threat of single-vendor hegemony over the M language. Its design–as well as some of the documentation included with the original sources–indicate that FreeM was originally targeted to the MS-DOS family of operating systems. It made use of a very limited subset of the C library, and included instructions for renaming the MS-DOS style 8.3 filenames in order to compile under UNIX.

At one point in FreeM’s early history, Shalom ported FreeM from MS-DOS to SCO UNIX, the SVR3-based descendant of Microsoft XENIX, now known as SCO OpenServer–a platform still supported by FreeM. This port brought support for the scoansi terminal type, including colors and X.364 control mnemonics.

Enter the GUMP

Around the time Shalom ha-Ashkenaz was developing FreeM, Richard F. Walters, a professor from U.C. Davis, conceived of the GUMP, an acronym standing for “Generic Universal M Project”. The GUMP, following the object-oriented programming craze of the 1990s, was intended to be a toolkit allowing M implementations to be built from discrete components with a well-defined and well-specified public interface among these components. These components included the global handler (supplying the database functionality), and the interpreter/compiler (responsible for implementing M language commands). The components would have been able to communicate over a network, or in-process on the same host, enabling distributed computing functionality.

Although the specification for the GUM interface to global handlers attained a reasonably well-specified level of completeness, and Larry Landis and others developed a mostly-complete implementation of a GUM global handler, none of the other envisioned components were ever completed, and specifically, the interpreter component was missing.

Shalom’s Gift

In July of 1998, Shalom ha-Ashkenaz donated the FreeM source code (then known as FreeMUMPS) to the M User’s Group-Deutschland (MUG-D), hoping that the community would take the nascent implementation from its infancy through to a state of production-ready completeness and robustness. Shalom also placed a few conditions on his gift: a public release could not be made until a substantial set of milestones were reached. Per his conditions, the FreeMUMPS project must:

• Implement the entirety of X11.1-1995
• Use Structured System Variables instead of VIEW commands/functions
• Raise the string size limits
• Implement MWAPI, OMI, X11 bindings, and GKS bindings
• Be substantially free of major bugs

Although MUG-D readily accepted the contribution of FreeMUMPS, the organization itself lacked the manpower and expertise to complete the implementation. Just as it is now, the intersection of M community members who know enough of the M language and C language to work on a project this ambitious was quite small.

Merging GUMP and FreeM

Very shortly after the contribution of FreeMUMPS to MUG-D, Richard F. Walters and a small team of developers and administrative staff who had been working on the GUMP assumed maintainership of the FreeMUMPS source code. This included representatives from the M Technology Association (an M vendor association having several foreign branches), the M Development Committee (the M standards organization hosting the ANSI/ISO standards for the M language, then sponsored by the M Technology Association), and others. The goals of this team were to:

• Meet Shalom’s requirements for a public release of FreeMUMPS
• Convert FreeMUMPS into the first interpreter component of the GUMP

During this era, Ronald L. Fox of Hawaii (who passed in 2010) ported FreeMUMPS from SCO UNIX to Red Hat 5 and glibc-6. Steve “Saintly” Zeck also attempted to rewrite the symbol table code to lift string size limits, David Whitten enhanced some of the implementation-specific extensions, and Larry Landis integrated Saintly’s symbol table work.

Early on in the GUMP maintainership of FreeM, the name of the implementation was changed from FreeMUMPS to Public Standard M, a change which was changed to Free Standard MUMPS and then FreeM when it was discovered that the PSM acronym was already in use for Patterson & Gray’s M implementation. Dr. Walters also received the implementation ID of 49 from then secretary of the M Development Committee, Don Piccone.

One of the contributors to FreeM at this stage–mainly in the area of M vendor routines–was Axel Trocha, who would go on to develop and maintain his own private fork of FreeM.

The GT.M Release

GT.M, standing for Greystone Technology MUMPS, is an M implementation that was released by Greystone Technology in 1986. Greystone was later acquired by Sanchez Computer Associates, which was in turn acquired by Fidelity National Information Services.

When GT.M was released under a free software license in 2000, it seemed to negate the entire raison d’etre for FreeM, as GT.M was a well-established, robust, and high-performance M implementation with which FreeM could not then compete. Unfortunately, at this time, the GUMP and FreeM projects lost all of their momentum, and new development along these lines rapidly ceased. The final GUMP team release of FreeM was 0.5.0. However, Axel Trocha’s private port would continue.

Axel Trocha’s FreeM Fork

After FreeM’s momentum ceased within the primary branch of development under Richard F. Walters’ leadership, Axel Trocha, an aforementioned contributor of M vendor routines and member of Dr. Walters’ team, continued development on the FreeM source code. Axel added many interesting features to the FreeM codebase, including:

• A native port to Microsoft Windows
• Compiling FreeM as an Apache web server module, allowing FreeM to be used easily for web development
• The ability to output HTML code in a heredoc-style format, with any line of code beginning with a left angle bracket being interpreted as HTML with support for interpolated M locals and globals
• Extensions allowing FreeM to be used as a command-line shell, along the lines of UNIX bash, Windows cmd.exe, etc.

Axel also maintains ownership of the freem.net Internet domain, and continued issuing public releases of his FreeM port on that site until sometime after 2003, at which point he took his port entirely private. Currently, freem.net is a blank page. However, Axel’s fork of FreeM continues to this day as the back-end database and programming environment for the www.elvenrunes.de website. I have communicated with Axel occasionally.

Resuming Primary Development Branch

In 2011, I downloaded the FreeM source code from the GUM Project’s SourceForge repository–dormant since 2000–and updated it just enough that it would compile and run on modern GNU/Linux systems. I also quickly updated FreeM to support terminal sizes larger than 80×24.

Taking Maintainership

In 2014, Larry Landis gave me administrator access to the GUMP repository, transferring maintainership of the primary branch of FreeM development to me. Since then, I have added many features and corrected many bugs, including:

• Adding support for proper namespaces, configured through /etc/freem.conf, which standardizes routine and global storage locations
• Adding support for Structured System Variables
• Adding support for the asynchronous event specification from the MDC Millennium Draft Standard
• Adding support for constants through the CONST keyword
• Adding a WITH keyword that allows you to specify an implicit prefix to all subsequent variable references
• Adding a runtime watch command (ZWATCH), which tracks changes to specified locals or globals
• Adding a ZASSERT command, which will fail with an error message if the following expression evaluates false
• Adding support for operators such as ++, –, +=, etc.
• Removing the Steve “Saintly” Zeck symbol table implementation, which was unreliable, and reverting to Shalom’s original implementation
• Adding support for the GNU readline library, with persistent command line history and editing, as well as some level of command-line completion
• Adding REPL-like functionality (in direct mode, any M expression beginning with a number will be prepended with an implicit WRITE)
• Adding transaction processing (a work in progress)
• Adding KVALUE and KSUBSCRIPTS
• Adding support for the M Windowing API (MWAPI), which is also a work in progress
• Adding the “fmadm” command-line utility, for database administration functions
• Adding support for after-image journaling and forward database recovery
• Adding support for TCP and UDP client sockets, for both IPv4 and IPv6
• Writing a texinfo manual, from which the HTML manual is derived
• Porting to Solaris/SPARC, Solaris/x86, Linux/s390x, Linux/armv6l, Linux/armv7l, SCO OpenServer 5.0.7, Tru64 UNIX/alpha, AIX/ppc, Mac OS X/x86, GNU HURD, Cygwin, NetBSD, FreeBSD, OpenBSD, and WSL1/2

I have also created the https://freem.coherent-logic.com website, where distribution downloads and documentation are readily available.

Future

FreeM is moving towards being a client-oriented desktop M implementation, for developing graphical user interfaces that will run on mobile and desktop devices.

I also intend to adopt the original vision of the GUMP team, dividing FreeM’s functionality into discrete components having a well-specified public interface, with the ability to run in distributed computing environments over a network.

FreeM’s mission is to advance the state-of-the-art in M implementations, and push the evolution of the language forward. Maintaining portability to as many vintage and modern UNIX systems as possible is held as a high priority, while portability of M routines and MDC standards compliance will be maintained only to the extent that it does not conflict with the primary goal of elegantly advancing the state-of-the-art and finding new audiences for the concepts originated by Neil Pappalardo and Octo Barnett in 1966.

FreeM is also strongly committed to free software principles, and is firmly aligned with the goals of the GNU Project and the Free Software Foundation, believing that the ethical concerns surrounding proprietary software are at least as important as the practical concerns of “open-source”.

FreeM is also being developed as a tool for enabling application development in worker/tenant cooperatives, and is committed to social justice and progressive principles.

If you are interested in FreeM, please see https://freem.coherent-logic.com for more information.

Why C is Almost Always the Wrong Choice

C has no true string data type.

The common arguments defending this as a feature rather than a shortcoming go something like this:

• Performance. The argument here is that statically-allocated, null-terminated char arrays are faster than accessing the heap, and by forcing the programmer to manage his own memory, huge performance gains will result.
• Portability. This one goes along the lines that introducing a string type could introduce portability problems, as the semantics of such a type could be wildly different from architecture to architecture.
• Closeness to the machine. C is intended to be as “close to the machine” as possible, providing minimal abstraction: since the machine has no concept of a string, neither should C.

If these arguments are true, then we shouldn’t be using C for more than a tiny fraction of what it is being used for today. The reality of these arguments is more like this:

• Performance: I’m a programmer of great hubris who actually believes that I can reinvent the manual memory management wheel better than the last million programmers before me (especially those snooty implementers of high-level languages), and I think that demonstrating my use of pointers, malloc(), gdb, and valgrind makes me look cooler than you.
• Portability: I’m actually daft enough to think that the unintelligible spaghetti of preprocessor macros in this project constitutes some example of elegant, portable code, and that such things make me look cooler than you.
• Closeness to the machine: I’ve never actually developed anything that runs in ring zero, but using the same language that Linus Torvalds does makes me look cooler than you.

The technical debt this attitude has incurred is tremendous: nearly every application that gets a steady stream of security vulnerability patches is written in C, and the vast majority of them are buffer overflow exploits made possible by bugs in manual memory management code. How many times has bind or sendmail been patched for these problems?

The truth is that most software will work better and run faster with the dynamic memory management provided by high-level language runtimes: the best algorithms for most common cases are well-known and have been implemented better than most programmers could ever do. For most corner cases, writing a shared library in C and linking it into your application (written in a high-level language) is a better choice than going all-in on C. This provides isolation of unsafe code, and results in the majority of your application being easier to read, and easier for open-source developers to contribute to. And most applications won’t even need any C code at all. Let’s face it: the majority of us are not writing kernels, database management systems, compilers, or graphics-intensive code (and in the latter case, C’s strengths are very much debatable).

The long and short of it is that most software today is I/O-bound and not CPU-bound: almost every single one of the old network services (DNS servers, mail servers, IRC servers, http servers, etc.) stand to gain absolutely nothing from being implemented in C, and should be written in high-level languages so that they can benefit from run-time bounds checking, type checking, and leak-free memory management.

Can I put out a CVE on this?