TypeRex development environment for Emacs

Most functionalities of TypeRex rely on some knowledge of the edited program (source files, libraries …) which should be specified in a very simple project file at the ``root'' of its source tree with name .typerex.

When TypeRex is invoked on a source file <file.ml>, it looks for file .typerex in the directory containing <file.ml, or its parent directories, back to the file system's root. This file is read at each command invocation (except syntax coloring and auto-completion) so modifications are taken into account immediately.

Syntax of .typerex files

The .typerex file should specify the set of directories to search for OCaml source files, and the set of directories to include in the load path (i.e., libraries). It is also possible to exclude some source files or whole compilation units, or to force files to be included whatever their extension (if any).

The syntax of the .typerex file is as follows:

        <project file> := <line>*
        <line> := <dirs>
                | I<dirs>
                | -<files>
                | IMPL <files>
                | INTF <files>
                | CMT <dir>
                | NOSTDLIB
                | #<comment>
         <dirs> := white-space-separated list of directories
         <files> := white-space-separated list of files

Meaning of project and library directories

Lines starting with I indicate that the specified directories are considered as library and not as project’s directories. The meaning of this distinction, which may change in the future, is currently the following:

• All source files (.ml, .mli, .mll, .mly) in a project directory are considered, whether they have corresponding compiled files (.cmi, .cmti, .cmt) or not, while compiled files without sources are ignored. This is exactly the opposite for libraries: all .cmi, .cmti, .cmt are considered, and uncompiled sources are simply ignored.
• Refactoring and browsing stops at the boundary of libraries, and no binding propagation is performed on the implementation of libraries (see the documentation for renaming and grep). This saves some computation time and is sound unless a library depends on the program (but the same question arises when the considered program is meant to be a library)

Pack modules (experimental)

Pack modules are understood by TypeRex if the source directories contain either a file

• pack.mlpack in the ocamlbuild format: a list of module names, possibly qualified (using /) by a path relative to the directory containing the pack.mlpack file, or
• pack.cmt, whose contents is a pack module (such as generated by ocp-type -pack. This option only works if the packed modules are in the same directory as the resulting pack, which is not the case when compiling with ocamlbuild.

Other options

CMT <dir>: It is possible to specify a CMT directory to search for .cmt(i) files when they are not found at the same place as the source files. This is needed if the build system moves the files around, but then if several modules (in different directories) have the same name, then outdated cmts won’t be assigned unless there only is one (matching with the source digest to resolve ambiguity).

IMPL <files>, INTF <files>: Use this to use TypeRex on source files with special extension (or none). The subdirectory containing these files still needs to be specified in the .typerex file. You will also have to enable TypeRex mode for those, either manually with M-x typerex-mode, or by extending auto-mode-alist or interpreter-mode-alist (see emacs.append).

NOSTDLIB: Do not implicitely include the standard library path. This is required when using \typerex\ on the OCaml compiler.

Fallback

If no specific configuration is provided, TypeRex considers as program the set of OCaml source files present in the directory containing the edited source file, with no libraries other than the OCaml stdlib.

Browsing OCaml code with TypeRex

Note on browsing commands: Each cursor motion incurred by a browsing action (except when clicking on grep results) is undoable with the standard Emacs shortcut (C-_).

Grep (C-o g / C-o t g)

(C-o g) display a click-able list (compile minor mode) of the connected definitions and occurrences of the identifier under the cursor. Invocation is the same as for renaming. Use (C-o t g) to grep the top-level module defined by the current file instead of an identifier.

Goto-definition (C-o d)

Places the cursor on the definition of the identifier under the cursor, opening the appropriate file in the current window if necessary.

Cycle-definitions (C-o a)

Places the cursor on an alternate definition of the identifier declaration under the cursor, opening the appropriate file in the current window if necessary. The typical effect is to switch between .ml and .mli files, but at the right place. This may be used only for top-level let-bindings (i.e. ’let’ and not ’let..in’, external statements, type declarations, exception declarations, and (recursive) module and module type declarations

Comment-definition (C-o c)

Display a description of the identifier under the cursor, with its lookup path, and any comments associated with it (in the sense of OCamldoc). The description is:

• the type, for a value or field
• the type declaration, for a type constructor
• the argument types (or "constant"), for a constructor or exception
• the module type, for a module or module type.

Refactoring OCaml code with TypeRex

Note on reverting and undoing: For all refactoring actions, the reverting of modified buffers and the undoing take one of the two following modes:

• If the modification is local to the current buffer, then it is reverted while keeping its history, and renamed if needed. This enables undoing with the standard emacs shortcut (C-_).
• If several files are modified, then all relevant buffers are reverted and their “local” undo-lists are cleared. Instead, the multiple-file modification is added to a global undo list and can only be undone with “C-o u”. A call to “C-o u” is also pushed onto the local undo lists of all modified buffers for convenience, so that (C-_) will also work.

Multiple-file undo (C-o u)

Undo the last multiple-file modification. Warning! This discards any subsequent modification of the affected files (a confirmation is asked in this case). All buffers editing one of the affected files are reverted, and their local undo lists are cleared (and then receive a single new “global-undo” item).

Renaming (C-o r / C-o t r)

Rename an identifier through an OCaml program.

(C-o r): The cursor must be placed on an identifier definition or reference (for example, a let binding or a pattern).

(C-o t r): Rename the top-level module defined by the current file instead of an identifier.

Renaming takes care of necessary propagation (e.g., when distinct values with the same name need to be renamed consistently because this name appears in a common interface), and capture is detected.

Renaming is implemented for: values, types, modules (non-recursive), module types, fields, constructors, and exceptions. Aa a convenience, a partial, unsound renaming of classes and class types is supported, but will miss all references to the “secondary” bindings of a class or class type, i.e., the closed and open types, and, for a class, the class type. Type variables, instance variable, methods, argument labels, and polymorphic variants are not supported.

The replacement is intended to be complete, up to the following known bugs:

• labels, e.g. renaming x in let x = .. in f ∼x yields f ∼y instead of f ∼x:y, and similarly with fun ∼x -> ..
• renaming a type which is in fact a class or a class type, or such that its renaming “propagates” to one (through module constraints and functor applications) will not rename the class or class type itself, or its references.

Note also the following limitation:

• including a module where an element is renamed with an afterwards masked name causes a capture error.

Reference pruning (C-o p)

Simplify the identifier references (longidents) by removing unnecessary qualification. This operation ranges on the current buffer.

Open elimination (C-o q) (for "qualify")

Remove (if possible) the open statement under the cursor and qualify the subsequent references as required. the let open .. in syntax is also supported by open elimination. This operation is currently slightly conservative, when the same module is opened again inside one of the items in the elimination scope (sub-modules, let open, and M.(...)) but a duplicate open at the same level will be correctly handled.

Syntax coloring

TypeRex implements its own version of syntax coloring. It is not yet fully stable, but already has some new features such as the inline marking of lexing errors (with help-info) and a smarter treatment of unterminated strings and comments.

Syntax coloring is not specialized for ocamlyacc/ocamllex files, but will usually give an acceptable result except for C-style comments.

Auto-completion

An experimental completion feature is proposed in typerex, currently only for identifiers (including methods, tags, labels and type variables). Once enabled, a menu of candidates is triggered when typing test or with the appropriate key (<`> by default) which also completes the longest common prefix. Other keys allow to select a candidate and insert it (<C-n>, <C-p>, and <\> by default), or to cycle between them (with <TAB>, see the Auto Complete Mode user manual).

The candidates computation takes into account the load path which is configured for the project, the open and include statements and unqualified identifiers until the current position in the edited file (in a very approximative and simplistic way) and the module qualification possibly prefixing the identifier to be completed.

TypeRex assumptions and supported code

Preprocessors

The browsing commands of TypeRex support ocamlyacc/ocamllex sources, and should work with other pre-processors which generate OCaml source files with appropriate line numbers directives. More precisely, the identifiers in a pre-processed source file which are actual identifiers of the source (i.e., not generated or transformed during pre-processing) should be OK to grep or jump from and to, if no generated code has the same location.

For ocamlyacc and ocamllex files, these "actual" identifiers correspond to the quoted OCaml code (between braces). Jumping to ocamlyacc entry points is not supported however, because the generated interface has no line number directives. Renaming may work in pre-processed or ocamllex/ocamlyacc source files, but has not been thoroughly tested. Other refactoring commands won’t work on ocamlyacc/ocamllex sources.

The camlp4 pre-processor (version 3.12.1) is supported, but only partially because its output is an ast which has insufficient location information (or a source file but without line numbers directives). ocp-type (or ocp-wrapper) can generate binary annotations with camlp4, but the result of TypeRex commands will sometimes be inaccurate on camlp4-processed sources (in particular, renaming should only be attempted for local or unexported value bindings).

Module packs

Module packing is supported to the extent of its treatment in the project description (see above), but is still experimental (and with the limitation that goto does not go through packs while grep does, as for include directives).

Dealing with outdated binary annotations

TypeRex is usually able to overcome sparse changes to the edited files (saved or not) w.r.t the last compiled version, and to recompute the right positions. This feature relies on the source snapshots which are embedded in .cmt files. This works also for refactoring commands, but in this case a confirmation will be asked before proceeding.

Permissive behavior

Some internal errors which could occur while processing some files (for example due to unhandled language features) may be caught and reported to the user (asking for a confirmation in the case of refactoring). This avoids giving up too soon on errors which are clearly harmless to a specific action.

Recovery and debugging

Except for restarting the server, this section is more intended to developing and debugging TypeRex.

Errors and server restart

If the OCP server crashes for any reason (or becomes crazy), it is possible to restart it using

 M-x ocp-restart-server

Logging

First, the TypeRex environment for Emacs will echo minimal information as messages in the mini-buffer, the history of which is kept in the special buffer *Messages*. This includes the startup procedure, feedback about the executed commands, and in case of unexpected error (which is a bug), a complete exception backtrace.

You may enable logging of debug information in ∼/.ocp-wizard-log by setting the ocp-debug variable to t (the trace will be huge and hard to read). The value of ocp-debug may also be a string, which is a comma-separated list (without whitespace) of uncapitalized module names in the TypeRex code.

Fail fast

In the context of debugging, it is usually easier to disable most exception handling to get a backtrace closer to the real problem. This can be done by setting ocp-dont-catch-errors to t. Note however that this will lead TypeRex to fail in cases which would normally have triggered tolerant behavior.

Profiling

TypeRex may dump profile information in ∼/.ocp-wizard-profile.out if ocp-profile is set to the name of a TypeRex command (see tools/ocp-wizard/main/owzServer.ml). Run profile ∼/.ocp-wizard-profile.out to generate a dot file (note that profile is not compiled or installed by default).