8 raco demod: Demodularizing Programs
The raco demodularize command (usually used with the shorthand raco demod) takes a Racket module and flattens its dependencies into a single compiled module, potentially with submodules. A file "‹name›.rkt" is demodularized into "‹name›_rkt_merged.zo".
See compiler/demod for an alternative way to use the demodularizer. Using #lang compiler/demod can cooperate with tools like raco make and raco setup, which is especially important for library modules (as opposed to end-user programs).
In its default configuration, raco demod supports flattening a module that represents an end-user program, so it discards all syntax and compile-time support in the module and its dependencies. Submodules are preserved, but their syntax and compile-time support are similarly discarded. The demodularized ".zo" file can be run by passing it as an argument to the racket command-line program, or it can be turned into an executable with raco exe.
Supply the -s or --syntax flag to preserve syntax and compile-time components of the module, so that it can be required the same as the original module. In that case, modules whose instances need to be shared with other librraies should be omitted from the demodularization using -x or --exclude-library. For example, -x racket/base is normally needed.
A large single module generated by the demodularizer may trigger size limits in the compiler that prevent whole-module optimizations. Set the PLT_CS_COMPILE_LIMIT environment variable to raise the limit, and check 'info logging at the 'linklet topic (e.g., set PLTSTDERR to info@linklet) for information about when compilation is restricted to smaller functions.
The raco demod command accepts these flags:
-o ‹file› —
writes the flattened module to ‹file› instead of "‹name›_‹ext›_merged.zo" for an input file "‹name›.‹ext›". -x ‹module-path› or --exclude-library ‹module-path› —
excludes the module in ‹module-path› from flattening, as well as all of its dependencies. -e ‹path› or --exclude-module ‹path› —
excludes the module in relative-file ‹path› from flattening, as well as all of its dependencies. -s or --syntax —
preserve syntax objects and phase levels greater than the run-time phase in the flattened result. Otherwise, only the run-time phase is preserved, and unused (or merely exported) definitions are pruned as possible. -M or --compile-any —
flattens the module to machine-independent form, instead of recompiling the flattened module to the current platform and Racket virtual machine; the output generated with -M loads more slowly than a machine-specific form, but raco decompile can show the flattened module in a format that is closer to source. See also --dump-mi. -r or --recompile —
(re)compiles the module to machine-dependent form after flattening; this mode is the default. --work ‹dir› —
uses ‹dir› to cache compiled modules in an intermediate form for flattening; using --work with the same ‹dir› for multiple uses of raco demod can greatly speed up demodularization, and since the cache is based on raco make, it works even with different input files or when modules to be flattened have changed since the last use of the cache. -g or --garbage-collect —
aggressively prunes definitions that are unreferenced on the assumption that the right-hand side of a definition has no side effect; due to that unchecked assumption, this conversion may not preserve the behavior of the input module. --dump ‹file› —
writes an S-expression representation of the module’s content to ‹file›, which can be helpful for understanding the content that is in the compiled flatten module. --dump-mi ‹file› —
writes a machine-independent form of the flattened module to ‹file›, the same as -M would write, but useful when -M is not used.
In addition to preserving submodules or of the source module, demodularization may introduce new submodules to hold portions of the flattening. The introduced submodules have names demod-pane- followed by an integer.
Changed in version 1.10 of package compiler-lib: Added -M/--compile-any,
--work, and support for Racket CS.
Changed in version 1.15: Added -x/--exclude-library,
-s/--syntax, --dump,
--dump-mi, and preservation of submodules.
8.1 Demodularizing Libraries
Demodularization of a library module with compiler/demod can create a module whose meaning is different than the original, since transitive dependencies (that are not specified as excluded) are copied into the flattened module. That copying can break sharing as needed for generated structure types or bindings. As a specific example, separate copies of racket/base will have distinct and incompatible implementations of keyword arguments for procedures.
To avoid problems, a good general strategy for flattening is
put all modules to be flattened into an "private/amalgam" subcollection of, where modules within "private/amalgam" can freely refer to each other;
create a module "private/amalgam-src.rkt" that requires modules from "private/amalgam" that need to be accessible from outside, where submodules in "private/amalgam-src.rkt" can provided different subsets of bindings from "private/amalgam";
create a module "mine/private/amalgam.rkt" as
#lang compiler/demod "amalgam-src.rkt" #:include (#:dir "amalgam") and
from outside "private/amalgam", use only "private/amalgam.rkt", perhaps via public modules that reprovide from "private/amalgam.rkt".
8.2 Language for Demodularizing
| #lang compiler/demod | package: compiler-lib |
A module using compiler/demod language compiles to a form that is the flattened (in the same sense as raco demod) version of a source module. See also Demodularizing Libraries.
A #lang compiler/demod module body starts with a module-path to flatten, it may be followed by options:
syntax
module-path option ...
option = mode | #:include (mod-spec ...) | #:exclude (mod-spec ...) | #:submodule-include (submod-spec ...) | #:submodule-exclude (submod-spec ...) | #:dump-demod file | #:dump-demod-mi file | #:no-demod mode = #:exe | #:dynamic | #:static mod-spec = #:module module-path | #:dir dir-path | #:collect collect-name submod-spec = identifier | (identifier ...)
The default mode is #:dynamic, which preserves syntax objects and compile-time support (like macros), but does not insist that all modules are copied into the flattened module. For example, if a module is referenced by a combination of submodules within module-path and no other module is reached by the same combination, then the benefit of copying the module into a submodule is limited. The #:static mode is like #:dynamic, but it ensures that all modules are included unless they are specified as excluded. The #:exe mode discards syntax and compile-time support, so it may be suitable for flattening a module that implements an end-user program.
When the #:include option is specified, then only modules covered by a mod-spec will be included in the flattened form; otherwise, all modules are candidates for inclusion. When the #:exclude option is specified, the modules covered by the mod-specs are excluded, even if they would otherwise be included according to a #:include specification. In other words, #:exclude is applied after #:include.
The #:submodule-include and #:submodule-exclude specifications are analogous to #:include and #:exclude, but for submodules immediately with #racket[_module-path]. If mode is #:exe, then the list of inclusions defaults to main and configure-runtime, otherwise the default is to have no specific inclusions.
A mod-spec either indicates a specific module with #:module or it indicates all modules in a given collection (and its subcollections) with #:collect. A collect-name is always a string with /-separated components.
If the #:no-demod option is specified, then mod-spec is not flattened, after all. Instead, the new module requires and reprovides mod-spec and each of its submodules. This mode is always used when a compiler/demod module is expanded, since expansion must produce syntax instead of a compiled module. This mode also may be useful during for development to avoid longer compile times from flattening or to check whether copying of modules for flattening creates any trouble.
A flattened module using compiler/demod has a build dependency on the original module, so a tool like raco make or raco setup will trigger reflattening if the source module changes, but the flattened module does not have a run-time or expand-time dependency on the original module. Modules excluded from the flattening via #:include and #:exclude remain as run-time and expand-time dependencies of the flattened module. In the default #:dynamic mode, additional dependencies may be preserved for modules that cannot be usefully merged, but #:static or #:exe mode copies even those modules into new submodules.
Compilation and expansion of a compiler/demod module creates a "compiled/demod" subdirectory in the same directory as the module. That subdirectory that holds freshly compiled versions of all dependencies of the flattened module in a form that is suitable for demodularization. This extra compilation is managed using compiler/cm, so changes to dependencies can be handled incrementally, but still separate from normal compilation of the dependencies.
Added in version 1.15 of package compiler-lib.