UW Cecil/Diesel/Vortex/Whirlwind Project Internal Documents

Project Ideas

• Some research project ideas

Using Cecil, Diesel, and/or Vortex

• Here is information on running Cecil, Diesel, and/or Vortex on UW CSE machines, with a minimum of muss or fuss
• Click here for a view of the Vortex compiler's memory usage.

Manuals

• Cecil

• A quick introduction to Cecil and its standard library: pdf, postscript
• The Cecil language reference manual: pdf, postscript, gzipped postscript
• The Cecil standard library manual: html, pdf, postscript, gzipped postscript

• Diesel

• A quick introduction to Diesel and its standard library: pdf, postscript
• The Diesel language reference manual (draft): pdf, postscript, gzipped postscript
  • See /projects/cecil/vortex-root/vortex/Diesel/notes/Diesel.txt for brief notes on how Diesel differs from Cecil.
• The Diesel standard library manual: html, pdf, postscript, gzipped postscript

• Vortex

• How to Use the Vortex Compiler: pdf, postscript, gzipped postscript
• Pragmas supported by the Vortex Compiler: pdf, postscript, gzipped postscript
• The Vortex RTL reference manual: pdf, postscript, gzipped postscript
• Performance Evaluation of Vortex-Compiled Programs: pdf, postscript, gzipped postscript

• Group memos: Assorted internal documentation

The sources to the above manuals, plus Makefiles to rebuild them properly, are in /projects/cecil/cecil/doc/manuals.

WASP/Cecil Mail Logs

• Threaded archive for wasp@cs list
  If you get prompted for a password, then
  1. Go to the wasp@cs MailMan page
  2. Use the "Unsubscribe or edit options" form at the bottom of the page
  3. Press the "Email My Password To Me" button, or just "Change Your Password" to something you can remember. No, I don't know how this is supposed to be secure.
• Threaded archive for cecil@cs list
• Old threaded archive for cecil@cs list
• wasp@cs MailMan page (use this to change your subscription options)
• wasp@cs list administration

Setting Up Your Own Vortex Working Tree

1. get in the Unix oo_gang and pl_gang groups (you can use chgrpsh +pl_gang +oo_gang to add these groups even before the default login groups propagates from the central system)
2. create a directory somewhere, e.g. on one of the /projects/swlab[1-8] or /projects/cecil* disks
3. cd into it
4. invoke /projects/cecil/vortex-root/vortex/bin/shell/internal/build-vortex-working-tree
   This creates a directory named vortex in the current directory, and populates it with your working tree. This step will will take a long time, checking out a lot of sources for Vortex, the Cecil and Diesel front-ends, and the Java front-end, and building the runtime libraries and the Cecil, Diesel, & Java libraries.  You also can pass arguments to the script to limit which front-ends are checked out; the default arguments are Cecil Diesel Java Rhodium. One thing to note: apparently /bin/sed and /uns/bin/sed do different things, and some of our scripts have been written assuming /bin/sed behavior, so make sure /bin is ahead of /uns/bin in your path.
5. we conventionally make a symlink ~/vortex to <the directory that you created in step 2>/vortex
6. If you use csh or tcsh as your shell, then add the following to your .cshrc:
   setenv VORTEX_HOME <the directory that you created in step 2>/vortex
   source ${VORTEX_HOME}/bin/shell/vortex.cshrc
   • If you use bash as your shell, then modify your .bashrc to do export NAME=DEF in place of setenv NAME DEF
7. source your .cshrc or .bashrc

If you're setting up a release on your home machine, you have a couple of options.

• If you'd like to have a duplicate copy of your UW working tree, with whatever edits you're in the middle of, then I suggest that you look into a file synchronization tool such as Benjamin Pierce's unison tool.  (I use this at home.)  It takes a bit of configuring to set up synchronization of only "real" files and not generated or temp files, but once it's set up, it works well.
• If you'd like to set up an independent working tree, separate from your working tree at UW, then you can do the following (assuming you have ssh, cvs, etc. on your home machine, and that your shell is csh-like):
1. create a directory somewhere
2. cd into it
3. check out a copy of the current code base:
   1. setenv CVS_RSH ssh (and add this to your .cshrc on your home machine)
   2. cvs -d :ext:yourname@trifid.cs.washington.edu:/projects/cecil/CVS co -P vortex-developer
      This should create a directory named vortex in the current directory, and populate it with your working tree sources.
4. copy the /projects/cecil/bin/yourarch/vortex (on Windows with a .exe extension) executable to vortex/bin/yourarch.
5. edit the vortex/bin/shell/vortex.cshrc and vortex/runtime/makefiles/{Arch,Vortex}Config.{default,yourarch} files if needed for your local configuration (see vortex/notes/INSTALL for more details).  to the extent possible, define variables in your .cshrc that will take the place of any settings in vortex.cshrc, so that you don't have to modify any files in ways that won't work when you want to cvs commit your home changes back to UW.
6. setenv VORTEX_BASELINE <the absolute path name to the current directory>/vortex (and add this to your .cshrc)
7. source vortex/bin/shell/vortex.cshrc (and add this to your .cshrc on your home machine)
8. run vortex/bin/shell/SETUP
   This should build all of the helper executables, libraries, and other support structures for the Vortex, Cecil, Whirlwind, Diesel, and Java systems.
9. we conventionally also make a symlink ~/vortex to ${VORTEX_HOME}

Hacking

We use CVS to manage our versions and concurrent work.  Check-out is done with normal cvs update -d -P commands.  We have a script vortex-commit that we use as a wrapper around cvs commit, which also checks out a copy of the latest CVS tree into the directory /projects/cecil/vortex-root/vortex (which we call "the baseline").

• oo_gang CVSWeb: Browseable view of our CVS repository.

If you will be looking at a lot of Cecil source code from the ${VORTEX_HOME}/Cecil/src/* directories, I recommend creating a directory ${VORTEX_HOME}/Cecil/src/links that contains symbolic links to all the .cecil files in the other directories, flattening out the directory hierarchy and putting all the source files in one convenient place. See ~chambers/cecil/links/MakeLinks for a script that (re)builds these links.  Likewise for Diesel source code in ${VORTEX_HOME}/Diesel/src/* directories; see ~chambers/diesel/links/MakeLinks for a script that (re)builds these links.

• Lore: Stuff you should know when hacking Vortex/Whirlwind.  NOTE: this is very out of date now, so probably don't bother.

You probably want to set up ssh public-key login on your research account, enabling you to log in to any other research machine w/o entering a password by hand, so that you can make easy use of Vortex's parallel make facilities.

Exercising, Testing, and Benchmarking Whirlwind

A bunch of .wil files exist for exercising and regression-testing the Whirlwind compiler.

Many smaller tests are in $VORTEX_HOME/Diesel/src/whirlwind/tests.

• One small test is fact.wil.
• A more serious regression test is test.wil, which includes a number of other test files from .../tests/{features,opts}.  You can run these other test files individually, or all of them as a group via test.wil.  Exercise WW both without and with optimizations, e.g. doing pmake and pmakeo2.  Please add your own test files as you implement new WIL features or new WW optimizations.

A number of Java benchmarks have been collected, in $VORTEX_HOME/Java/src.  These benchmarks can be translated into WIL by going to their $VORTEX_HOME/Java/src/<benchmark-name>/wil subdirectory and executing the Jil-em script.  (The script is controlled by the Files, Classpath, MainName, and AppName files.  If you want to add your own Java benchmarks, which use only the subset of the Java 1.0.2 library that we support, create these files for your program and then run Jil-em.)  One that comes pre-translated into WIL is Towers of Hanoi, in $VORTEX_HOME/Java/src/example.  Its Java source is in .../example/src, the compiled Java .class files are in .../example/classes, and the WIL files translated from the .class files are in the .../example/wil.  The Java standard library classes also are already translated to WIL; those files are in $VORTEX_HOME/Java/wil-stdlib.  (WIL translations of all the benchmarks and the standard library classes can alternatively be found in /projects/cecil/WIL-src/Java/{<benchmark-name>,wil-stdlib}.)

To use WW to compile one of these benchmarks, you should put the directory containing the benchmark's .wil files and the directory containing the Java standard library classes' .wil files into WW's source path, e.g.:

Whirlwind> append source_paths $VORTEX_HOME/Java/src/<benchmark-name>/wil
Whirlwind> append source_paths $VORTEX_HOME/Java/wil-stdlib

Then you can compile the Java benchmark, e.g.:

Whirlwind> pmakeo2 <benchmark-name>.wil

An even bigger regression test is the make-whirlwind-profile script, which takes as its argument the path to the WW compiler executable to test, and uses it compile test.wil, towers.wil, a small Diesel program, and others, under many different optimization and target-code configurations (e.g. w/ and w/o RTA, for C or C-- output, w/ and w/o Rhodium opts, etc.).  This is the main check to see whether it's safe to CVS-commit a new WW version.  It also is the script used when building a new whirlwind.nCCP profile.

Finally, scripts exist for doing performance measurements of Whirlwind.  The main scripts are $VORTEX_HOME/Java/bin/shell/{Build,Run}-Whirlwind-Java-Benchmarks.  They should be run in a directory containing a symbolic link named whirlwind to the WW executable to use.  The Build script compiles all the Java benchmarks under a variety of configurations (and takes a long time); you can run this script repeatedly to make sure that things are up to date, and it won't recompile anything it's already compiled.  It writes a summary of its progress to a file named buildLog, but I usually redirect all its output to a log file too.  The Run script will then invoke the compiled benchmarks, running them 11 times and saving the resulting execution times to a file named whirlwind-java-data.db, and finally print out results using median execution times for all benchmarks and configurations, suitable for copying into a paper.  To get reliable timings, this script should be run on an otherwise unloaded, fast machine.  Again, I usually redirect this script's output to the same log file.  These scripts make use of other scripts, $VORTEX_HOME/bin/shell/{buildConfig,runConfig,showData}, which are controlled by the $VORTEX_HOME/bin/shell/configs.perl file.  You can add new optimization configurations for testing, or new named groups of benchmarks or configurations, by editing configs.perl, and optionally editing the {Build,Run}-Whirlwind-Java-Benchmarks scripts, e.g. as I've done to add the -cha option to the scripts that exercises the CHA-related configurations.  You also can explore the other options that are available to the underlying buildConfig, runConfig, and showData scripts by looking at the end of the configs.perl file.

Making a Release

See the file ${VORTEX_HOME}/notes/internal/for-release for instructions on how to make a new release.