Chapter 2: The Wedger Elves


The Wedger Elves run mosflm. They process one (and only one) "wedge" of x-ray diffraction image data. They are most familiar with images collected with R-axis, ADSC Quantum 4, and MAR type detectors, but they still support any data that can be read with mosflm. Wedger Elves are capable of everyting you need to do to process a wedge, from indexing to scaling and mergeing. If all you have is single-wedge data, then Wedger Elves should be all you need to get a completely optimized data set out of your diffraction images, and it's all in one file!

To use Wedger Elves, all you need is a mosflm executable for your computer system, and the Wedger Elves script. Wedger Elves are the only Elves that can funciton without CCP4. All the indexing, refinement, and measurment features will work even if CCP4 is not installed on your computer system, but scaling and mergeing, which require scala and truncate, will not work without CCP4. In the absence of CCP4, the mtz file (CCP4 format) produced by mosflm will be converted by Wedger Elves into a text file that you can then reformat and use in a non-ccp4 scaling routine, if you want.

Wedger Elves are meant to be a free-standing, adaptive and absolutely intuitive interface to mosflm. Even if you have never heard of mosflm before, as long as you have a general idea about what oscillation data is, you shouldn't have any trouble using Wedger right "out of the box." If you havn't installed the mosflm executable in your path, Wedger Elves will go looking for it. If you don't tell them where your data are, they will go looking for that too. Simply type in what you want the Wedger Elves to do on their command-line, in your own words, and they will work very hard to make it happen for you. In the end, Wedger Elves will leave you with a mosflm script for measuring all the spots on your diffraction images, and a scala script for scaling and mergeing them together.

2.1 Autoindexing

Elves support many autoindexing options:

  1. Interactive real-space (DPS) indexing (recommended)
  2. Interactive "refix" indexing (deprecated, use #1)
  3. background "refix" indexing (best with space group and unit cell provided)
  4. inherit orientation from another wedge (recommended for multi-wedge data from same xtal setting)
  5. inherit orientation from denzo output (if you've already indexed with denzo)
  6. background denzo indexing (requires a denzo licence)

The DPS indexing (#1) is, in my experience, the most reliable method of getting your first orientation. However, with current versions of mosflm, there is no way for Elves to do it for you! You must use the mosflm GUI. Before launching the GUI, Wedger Elves will provide you with a short list of simple instructions as to how to use the interactive mosflm indexing. At best, you will only have to push one button in the mosflm GUI window: "Autoindex", and type in your choice of space group.

If you have more than one image, you will always get better results by using spots from several available images. The more widely-separated they are in phi, the better. To do multi-frame indexing, push the "Read Image" and "Find Spots" buttons, and follow the directions you see to pick spots. Once you have picked a few images (2 is fine, 4 is usually plenty) push the "Autoindex" button. As long as your crystal isn't twinned or split into more than one lattice, this should always work. Just make sure you have at least a few hundred real spots picked. If mosflm is having trouble finding your spots (a common problem with really weak diffraction), you can easily pick them manually by answering "Y" to the "Do you want to find spots manually ? (N)" question. Then you click on (or near) the spots you want to use for indexing. Kind of a pain, but it doesn't take too long.

Once you have pressed the "Autoindex" button, followed the on-screen instructions, and selected an orientation solution, you should hit the "Predict" button to make sure your spots are covered by prediction boxes. If they're not, you probably either picked a bad solution, have an incorrect beam center, or your crystal is cracked (more than one lattice).

If your beam center is wrong, and you have ice rings, you can re-define the beam center using the "Fit Circle" button (not shown above).

Elves can use the old "refix" reciprocal-space indexing algorithm either interactively with you, or all by themselves, in the background, but they need two or more widely-separated frames to do it sucessfully. If, for some reason you can't do any of these, (perhaps you do not have graphics display privileges), Wedger Elves will go and try to find a copy of denzo to index your first frame with. Wedger Elves can also be given a denzo output file, and read the crystal orientation (and the rest of the crystal parameters) from it.

Once they have an orientation, you do not, necessarily, need the graphics anymore.

Unlike denzo, mosflm keeps the crystal orientation in a separate file, called the orientation matrix file. This may seem a little strange to people who are used to denzo, but it is more convenient for setting up multi-wedge data processing. All you have to do is copy the matrix file around. Better yet, let the Wedger Elves worry about it.

Remember, if you symmetry is trigonal or higher (that is, if you have two cell dimensions that are very near the same length) it is not a good idea to index more than once. You should index one wedge, and then just use that orientation for the rest of the wedges. Otherwise, your two equivalent cell edges will be randomly assigned to a or b, and, yes, a and b are different!

2.2 Indexing with Denzo

I am frequently asked why Wedger Elves use Andrew Leslie's mosflm, and not HKL software's denzo. Discussions on this matter tend to produce far more heat than light, so I will be brief.

There are three reasons why Wedger Elves do not (directly) support denzo.

  1. denzo is not free, so Elves wouldn't be free either if they required denzo.
  2. denzo, internally, is not as automated as mosflm, making it more complicated for Elves to use it.
  3. my personal experiments with denzo and mosflm imply that they are, essentially, equivalent.

Nevertheless, Wedger Elves can be made to use an installed and licenced copy of denzo to index your first image, and then import the resulting crystal-orientation solution into automated processing with mosflm. (N.B. Wedger Elves cannot use denzo to index images from a detector for which you do not have a valid license.)

To use denzo for autoindexing in Wedger Elves, do the following:

  1. Answer "no" to the "Do you want to autoindex INTERactively (requires graphics)?" question (if you see it).
  2. Ctrl-C out of the "examining frames named ..." prompt (background mosflm indexing).
  3. Answer "no" to the "launch mosflm and do some autoindexing now?" question.
  4. Answer "yes" to the "Try using denzo?" question. (this will only appear if you have a denzo licence file)

Wedger Elves will then start looking for a copy of denzo that can index your frame to their satisfaction. The auto.dat file they produce can also be used by you as a starting point for denzo autoindexing should Wedger Elves fail.

2.3 Collection Strategy

After collecting and indexing your first frame, the first thing you usually want to know is what range of phi values will give you a complete data set in the least time. Fortunately, mosflm has a strategy package build right into it. Elves will, by default run three strategy calculations:

  1. predict the completeness of the current wedge (usually pretty bad for only one frame!)
  2. find the best additional wedge to collect (given that you've already collected some data)
  3. find the best, overall collection strategy.

Elves will also warn you if your crystal has a unique crystal axis too close to the phi axis (always true in P1). If this happens, you should tilt your crystal a bit (usually 20-30 degrees perpendicular to the phi axis) and then index again. Elves also warn about potentially overlapping spots (dependent on the mosaicity you give them), and suggest a range of oscillations widths to use. Many crystal settings will have a "problem spot" (usually close to a zero-plane) where you need tiny oscillation widths to avoid overlaps, but can get away with fairly big oscillations for the rest of the wedge. Wedger Elves will leave a 360-degree analysis of potential overlaps in logs/strategy.log.

In case you want to do more complex strategy calculations yourself, Wedger Elves provide you with a strategy.inp file. This is a mosflm input file. To use it, edit it, and then feed it to mosflm: 

unix% mosflm
MOSFLM==> @strategy.inp

then you can start playing with the strategy functions. To find out more about how to use strategy, see the mosflm manual.

2.4 Refining Camera and Crystal Parameters

All modern data reduction programs refine the camera and crystal parameters you input into them. However, not all of them take advantage of the fact that some parameters, although "refinable" should be constant for a given experiment (the unit cell, for example).

The refinement strategy implemented by Wedger Elves is derived from the recommended procedure outlined in the mosflm manual:

  1. refine the cell and orientation against a subset of the images in the wedge
  2. repeat the above refinement if parameters changed significantly
  3. refine against all images, measuring spot intensities along the way
  4. repeat the whole measurement if parameter values shift significantly

The initial refinement procedure is done on a subset of frames (several, contiguous bunches, evenly spaced across the wedge), and only partials are measured for the sake of speed. This means that, in the first few mosflm runs conducted by Wedger Elves, several blocks of frames are "skipped" as mosflm moves through the wedge.

Wedger Elves will suggest repeating the refinement whenever they see that the output, refined parameters differ significantly from the ones that were input at the beginning of the refinement. Changes of >0.03 degrees are considered significant, since they represent (approximately) an error of ~3% in the partiality of measured spots.

To learn more about refinement in mosflm, have a look at the mosflm manual.

2.4.1 Conceptual shifts for denzo users

Crystallographers accustomed to using denzo, where each image's parameters are refined independently, may be unfamiliar with mosflm's approach to data processing. While processing with denzo, one can usually see slow, systematic variations in the unit cell over phi, and these variations are accompanied by a corresponding change in the xtal-to-film distance (which is highly correlated with the unit cell). This presents some problems when importing denzo data into CCP4 (see the scala manual for details). In the HKL package however, these variations are "cleaned up" in scalepack using postrefinement, which is the traditional way of applying project-wide constraints to an x-ray data set. However, (in my experience) fixing the unit cell in denzo (once I know what it is), results in "better" statistics in the next scalepack run. HKL's newest package, HKL2000 includes such "postrefinement" information in the measurement process automatically.

Mosflm (and Wedger Elves) take a similar approach. "post"refinement is done during the measurement procedure. This means that mosflm itself refines your mosaicity, unit cell, and other parameters, before the spots are even measured. Mosflm can also accumulate spot profiles over several, adjacent images, a procedure which is sometimes called 3D profiling.

2.5 Measuring intensities

There are two major modes of operation in mosflm: refinement (described above), and full intergration. Full integration post-refines the data in a short window (usually around 5-10 frames wide), integrates all the spot data, and then writes the intesisites to the output file.

Wedger Elves HOWTO

Okay, so how do you use Wedger Elves? Well, a copy of the Wedger Elves script should be placed in a directory called Elfsheim when you run the Elves main program. However, a separate copy of Wedger Elves .

Wherever you put the Wedger script, all you have to do is run it with a few words about what you want to do on the command line:

unix% Wedger Elves, process the data in /data/jamesh/frames

There are also a few special command-line options to Wedger Elves, mainly for customizing behavior in non-interactive, background runs:

-burst 3     do 3 rounds of refinement, integrate, and then scale and merge, regardless of wether Elves think things have converged or not.
-auto        force automatic mode (they will not ask you any questions)
-1            run once, and then exit
-norun        just write the script, but don't run it.
integrate     write an integrating script this time
no integrate write a non-inregrating (refine only) script this time
-new         indicate that this is a new run (don't look around the disk for old data)
-nice        tell Wedger Elves to "nice" their mosflm jobs


As of version 1.2, the Wedger Elves command-line is interpreted differently from text entered on the Wedger Elves internal prompt. This will be fixed in the future, but users of the alpha version should bear in mind that "control" words (such as the ones listed above) are not usually understood in Wedger's interactive command prompt, and that crystallographic parameters, such as the mosaicity, are not usually understood on the command line.

As a rule of thumb, it is best to communicate experimental parameters to Wedger Elves by writing them down into a text file, and then providing the name of that file on the command-line.

Flow-of-control commands, like -norun, -burst, and -new only have meaning at the beginning of the Wedger run anyway (if you decide not to run the script while Wedger is running, just type "exit").

If worse comes to worst, you can always communicate parameters to Wedger Elves by killing Wedger, editing the script, and then running: Wedger This will usually give them the right idea.

Scripts written by Wedger Elves

Like all Elves, Wedger Elves work by writing and editing shell scripts for mosflm and CCP4 programs. These scripts have been designed to be easy to read and edit. Some of them also contain a "smart setup" section to make them as flexible as possible in their unmodified form. Some examples of these scripts are shown here, with a brief description of how to run them:

Back to the Elves Manual Table of Contents.

This page is not finished. It will never be finished, and neither will yours. Admit it.

James Holton <>