*** Chapter 1: INTRODUCTION *** *** 1 OVERVIEW *** FERRET is an interactive computer visualization and analysis environment designed to meet the needs of oceanographers and meteorologists analyzing large and complex gridded data sets. "Gridded data sets" in the FERRET environment may be multi-dimensional model outputs, gridded data products (e.g. climatologies), singly dimensioned arrays such as time series and profiles, and for certain classes of analysis, scattered n-tuples (optionally, grid-able using an objective analysis procedure provided in FERRET). FERRET accepts data from ASCII and binary files, and from two standardized, self-describing formats. FERRET's gridded variables can be one to four dimensions - usually (but not necessarily) longitude, latitude, depth, and time. The coordinates along each axis may be regularly or irregularly spaced. FERRET was developed by the Thermal Modeling and Analysis Project at NOAA/PMEL in Seattle to analyze the outputs of its numerical ocean models and compare them with gridded, observational data. Model data sets are often multi-gigabyte in size with mixed 3 and 4-dimensional variables defined on staggered grids. FERRET offers the ability to define new variables interactively as mathematical expressions involving data set variables. Calculations may be applied over arbitrarily shaped regions. FERRET provides fully documented graphics, data listings, or extractions of data to files with a single command. Without leaving the FERRET environment, graphical output may be customized to produce publication-ready graphics. Animations are also available. FERRET is supported on a variety of Unix worstations. And, a point-and-click interface is under development at the time of this writing. FERRET is available at no charge from anonymous FTP [node abyss.pmel.noaa.gov] or from Mosaic [URL http://www.pmel.noaa.gov/]. For users with access to NCSA Mosaic on the Internet, FERRET may be perused in hypertext. *** 2 GETTING STARTED *** A quick way to get to know FERRET is to run the tutorial provided with the distribution. % ferret yes? GO tutorial If FERRET is not yet installed consult Chapter 7. (The tutorial is also available through Mosaic.) The tutorial demonstrates many of FERRET's features, showing the user both the commands given and FERRET's textual and graphical output. You may find the explanations, terms and examples in this manual easier to understand after running the tutorial. *** 2.1 Concepts *** Words in bold below are defined in the glossary of this manual. In FERRET all variables are regarded as defined on grids. The grids tell FERRET how to locate the data in space and time (or whatever the underlying units of the grid axes are). A collection of variables stored together on disk is a data set. To access a variable FERRET must know its name, data set and the region of its grid that is desired. Regions may be specified as subscripts (indices) or in world coordinates. Data sets, after they have been pointed to with the SET DATA command, may be referred to by data set number or name. Using the LET command new variables may be created "from thin air" as abstract expressions or created from combinations of known variables as arbitrary expressions . If component variables in an expression are on different grids, then regridding may be applied simply by naming the desired grid. The user need never explicitly tell FERRET to read data. From start to finish the sequence of operations needed to obtain results from FERRET is simply: 1) specify the data set 2) specify the region 3) define the desired variable or expression (optional) 3) request the output For example (Fig. 1), yes? SET DATA coads !global sea surface data yes? SET REGION/Z=0/T="16-JAN-1982"/X=160E:160W/Y=20S:20N yes? VECTOR uwnd,vwnd !wind velocity vector plot *** 2.2 Sample sessions *** This section presents a number of short FERRET sessions which demonstrate common uses. Data sets used in these sessions and throughout this manual are included with the distribution. If FERRET is installed on your system, you can duplicate the examples shown. *** 2.2.1 Accessing a formatted data set *** In this sample session, the data set "monthly_navy_winds" is specified and certain aspects of it are examined. The command SHOW DATA/FULL displays the variables in "monthly_navy_winds" and where on each axis they are defined. SET REGION specifies where in the grid the user wishes to examine the data. VECTOR produces a vector plot of the indicated variables over the specified region. % ferret yes? SET DATA monthly_navy_winds !specify data set yes? SHOW DATA/FULL !what's in it? currently SET data sets: 1> /home/e1/tmap/fer_dsets/descr/monthly_navy_winds.des (default) FNOC 2.5 Degree 1 Month Average World-wide Wind Field diagnostic variables: NOT available name title I J K L UWND ZONAL WIND 1:144 1:73 1:1 1:132 ... on grid FNOC251 X=18.8E:18.8E(378.8) Y=91.2S:91.2N VWND MERIDIONAL WIND 1:144 1:73 1:1 1:132 ... on grid FNOC251 X=18.8E:18.8E(378.8) Y=91.2S:91.2N time range: 16-JAN-1982 20:00 to 17-DEC-1992 03:30 yes? SET REGION/X=170E:110W/Y=5S:5N ! geographical region yes? VECTOR uwnd,vwnd !make a vector plot yes? QUIT *** 2.2.2 Reading an ASCII data file *** Many examples of accessing ASCII data are available later in this manual. See Chapter 2 "Data Sets". The simplest access, one variable with one value per record, looks like this: % ferret yes? FILE/VARIABLE=v1 snoopy.dat yes? PLOT v1 yes? QUIT *** 2.2.3 Using viewports *** The command SET VIEWPORT allows the user to divide the output graphics "page" into smaller display viewports. In this sample session, we create 2 plots in 2 halves of a window (Fig. 2): % ferret yes? SET DATA coads_climatology yes? SET REGION/X=160E:130W yes? SET REGION/Y=-10:10/L=5 yes? SET VIEWPORT upper yes? CONTOUR sst yes? SET VIEWPORT lower yes? CONTOUR airt yes? QUIT *** 2.2.4 Using abstract variables *** Abstract variables (expressions that contain no dependencies on disk-resident data) can be easily displayed with FERRET. See Chapter 3 section "Abstract variables" for several examples and detailed information. For example, a user wishing to examine the function SIN(X) on the interval [0,3.14] might use (Fig. 3): % ferret yes? PLOT/I=1:100 sin(3.14*I/100) yes? QUIT *** 2.2.5 Using transformations *** A transformation is an operation performed on a variable along a particular axis and is specified with the syntax "@trn". See Chapter 3 section "Transformations" for detailed information. A user may wish to look at ocean temperatures averaged over a range of depths. In this sample session, we look at temperature averaged from 0 to 100 meters of depth using a data set which has detailed resolution in depth (Fig. 4). We plot the data along longitude 160 west from latitude 30 south to 30 north. % ferret yes? SET DATA levitus_climatology yes? SET REGION/Y=30s:30n/X=160W yes? PLOT temp[Z=0:100@AVE] yes? QUIT *** 2.2.6 Using algebraic expressions *** See Chapter 3 section "Expressions" for a description of valid expressions. In this example, the data set contains raw sea surface temperatures, air temperatures, and wind speed measurements. We wish to look at a shaded plot of sensible heat at its first timestep (L=1) (Fig. 5). We specify a latitude range and contour levels. % ferret yes? SET DATA coads_climatology !monthly COADS climatology yes? LET kappa = 1 !arbitrary yes? LET/TITLE="SENSIBLE HEAT" sens_heat = kappa * (airt-sst) * wspd yes? SHADE/L=1/LEV=(-20,20,5)/Y=-90:40 sens_heat yes? QUIT *** 2.2.7 Finding the 20 degree isotherm *** Isotherms can be located with the "@LOC" transform, which returns the axis location where the value of the argument of @LOC first occurs. Thus, "TEMP[Z=0:200@LOC:20]" locates the first occurrence of the temperature value 20 along the Z axis, scanning all the data between 0 and 200 meters. A session examining the 20 degree isotherm in mid-Pacific ocean data (Fig. 6): % ferret yes? SET DATA levitus_climatology yes? SET REG/Y=10s:30n/X=140E:140W yes? PPL CONSET .12 !label size yes? CONTOUR temp[Z=0:200@LOC:20] yes? QUIT Note that the transformation @WEQ could have been used to display ANY variable on the surface defined by the 20 degree isotherm. *** 3 COMMON COMMANDS *** A quick reference to the most commonly used FERRET commands (typing "SHOW COMMANDS" at the FERRET prompt lists all commands): Command Description SET DATA_SET names the data set to be analyzed SHOW DATA produces a summary of variables in a data set SHOW GRID examines the coordinates of a grid SET REGION sets the region to be analyzed LIST produces a listing of data PLOT produces a plot CONTOUR produces a contour plot (/FILL for color-filled contour plot) VECTOR produces a vector arrow plot SHADE produces a shaded-area plot STATISTICS produces summary statistics about variables and expressions LET defines a new variable SAVE saves data in NetCDF format GO executes FERRET commands contained in a file Information on all FERRET commands is available in Part II, Commands Reference, of this manual. *** 4 SYNTAX *** Commands in program FERRET conform to the following template: COMM [/Q1/Q2...] [SUBCOM[/S1/S2...]] [ARG1 ARG2 ...] [!comment] where COMM is a command name yes? LIST Q1... are qualifiers of the command yes? CONTOUR/SET_UP SUBCOM is a subcommand name yes? SHOW MODE S1... are qualifiers of the subcommand yes? SET LIST/APPEND ARG1... are arguments of commands yes? CANCEL MODE INTERPOLATE notes... - Items in square brackets are optional. - One or more spaces or tabs must separate the command from the subcommand and from each of the arguments. Spaces and tabs are optional preceding qualifiers. - Multiple commands, separated by semi-colons, can be given on the same line. - Command names, subcommand names, and qualifiers require at most 4 characters. (e.g.yes? CANCEL LIST/PRECISION is equivalent to yes? CANC LIST/PREC) - Some qualifiers take an argument following " = " (e.g. yes? LIST/Y=10S:10N). *** 5 GO FILES *** GO files are files containing FERRET commands. They can be executed with the command "GO filename". Throughout this manual, these files are referred to as GO scripts or journal files (the file names end in *.jnl). There are two kinds of GO files provided with the distribution (differing in function, not form) - demos and tools. *** 5.1 Demonstration files *** Demonstration GO files provide examples of various FERRET capabilities (the tutorial is such a script) . The demonstration GO files may be executed simply by typing the FERRET command yes? GO demo_name example: yes? GO spirograph_demo Below is a list of the demo files provided (located in directory $FER_DIR/examples). The Unix command "Fgo demo" will list all GO scripts containing the string "demo". Name Description tutorial brief tour through FERRET capabilities topographic_relief_demo global topography coads_demo view of global climate using the Comprehensive Ocean-Atmosphere Data Set levitus_demo T-S relationships using Sydney Levitus' climatological Atlas of the World Oceans fnoc_demo Naval Fleet Numerical Oceanography Center data vector_demo vector plots wire_frame_demo 3D wire frame representation custom_contour_demo customized contour plots viewports_demo output to viewports multi_variable_demo multiple variables with multiple dependent axes objective_analysis_demo interpolating scattered data to grids polar_demo polar stereographic projections log_plot_demo log plots using PPLUS in FERRET depth_to_density_demo contour with a user-defined variable as an axis file_reading_demo reading an ASCII file regridding_demo tutorial on regridding data mathematics_demo abstract function calculation statistics_demo probability distributions spirograph_demo for-fun plots from abstract functions splash_demo for-fun mathematical color shaded plots *** 5.2 GO tools *** GO tools are scripts which contain FERRET commands and perform dataset-independent tasks. For example, "GO land" overlays the outline of the continents on your plot. (Note: In order for FERRET to locate the GO scripts, the environment variable FER_GO must be properly defined. See Chapter 7 "Computing Environment" for guidance.) The Unix command Fgo has been provided to assist with locating tools within the Unix directory hierarchy. For example, % Fgo grid displays all tools with the substring "grid" in their names % Fgo '*' displays all GO tools and demonstrations Below is a table of the tools provided with your FERRET installation. Some tools accept optional arguments to control details. Use Fgo -more script_name for details on a script. Tool name Description OVERLAYS land [color] overlays continental boundaries (color controls) bold_land overlays darker continental boundaries fland [res color] overlays filled continent masses (color and resolution controls) graticule sets the plot axis style to use a graticule (rather than tics) tics resets the plot style to use axis tics (rather than a graticule) gridxy overlays a "graticule" labelling the I,J subscripts gridxz overlays a "graticule" labelling the I,K subscripts gridxt overlays a "graticule" labelling the I,L subscripts gridyz overlays a "graticule" labelling the J,K subscripts gridyt overlays a "graticule" labelling the J,L subscripts gridzt overlays a "graticule" labelling the K,L subscripts box draws a box at the specified location on the plot ellipse draws an ellipse at the specified location on the plot MATHEMATICAL frequency_histogram makes a frequency distribution plot (histogram) of data ts_frequency creates a 2-variable histogram (typically an oceanographer's TS density diagram) polar defines R and THETA from X and Y to perform (limited) polar plots regressx defines variables for linear regression along X axis regressy defines variables for linear regression along Y axis regressz defines variables for linear regression along Z axis regresst defines variables for linear regression along T axis unit_square sets unit square as default for abstract variables variance defines variables to compute variances and covariances var_n refines TVARIANCE with corrected n/n+1 factors dynamic_height defines FERRET variables for dynamic height calculations SAMPLE DISPLAYS line_samples draws specimens of the available line styles line_thickness draws examples of pen color/thickness styles in PPLUS fill_samples draws specimens of the available fill styles show_symbols draws specimens of the default symbols show_88_syms draws specimens of all 88 PPLUS symbols GRAPHICS bar_chart makes a color-filled bar chart from a line of data bar_chart2 makes a bar chart using hollow rectangles centered_vectors makes a vector plot with coords at vector midpoints scattered_vectors makes a vector plot from an ASCII file: x,y,u,v stick_vectors makes a stick vector plot of a line of U,V values PLOT APPEARANCE magnify [factor] increases the data plotting area (area inside the axes) unmagnify restores the plot origin and axis lengths to default values black sets video background to black, foreground to white white sets video background to white, foreground to black bold sets up PLOT+ and FERRET to produce bolder-looking plots unbold resets plot environment to normal after "GO bold" unlabel [label #] removes a specified (numbered) PPLUS movable label box_plot produces a plot with "bare" axes (no tics, no labels) COLOR try_palette [pal] displays palette appearance for various numbers of color levels try_centered_palette displays centered palette appearance for various numbers of levels exact_colors sets up FERRET and PPLUS to modify individual colors in a color palette squeeze_colors modifies a color palette by squeezing and stretching the color scale MULTIPLE X AND Y AXES (run demo: yes? GO multi_variable_plots) left_axis_plot plots a single variable preparing for a 2nd axis on the right right_axis_plot overlays a plot of a single variable using an axis on the right multi_xaxis_plot1 draws a plot formatted for later overlays using multiple X axes multi_xaxis_overlay overlays a variable with a distinct X axis multi_yaxis_plot1 draws a plot formatted for later overlays using multiple Y axes multi_yaxis_overlay overlays a variable with a distinct Y axis VIEWPORTS landscape set up for 8.5 x 11 page: landscape orientation landscape1x2 8.5 x 11 page - two viewports stacked landscape2x1 8.5 x 11 page - two viewports side-by-side landscape2x2 8.5 x 11 page - 4 viewports/reduced labels landscape3x2 8.5 x 11 page - 6 viewports/reduced labels portrait sets up for 8.5 x 11 page: portrait orientation portrait1x2 sets up for two viewports on an 8.5 x 11 page portrait1x3 sets up for three viewports on an 8.5 x 11 page portrait1x4 sets up for four viewports on an 8.5 x 11 page POLAR STEREOGRAPHIC PROJECTIONS (run demo: yes? GO polar_demo) convert_to_polar_2d extracts (sample) data for a 2D polar plot polar_2d produces a 2D polar stereographic plot polar_fland overlays solid-filled continents on a polar plot polar_grid overlays "quick" radial longitude and curved latitude lines polar_grid_fancy overlays "fancy" radial longitude and curved latitude lines polar_land overlays the continental outlines on a polar stereographic plot polar_map_inv_eqns defines the equations used for polar projections polar_vector produces a 2D polar stereographic vector plot polar_vs performs a polar PLOT/VS of 2 variables: lat long projected_map_grid defines the map grid for a projected plot TESTS test tests proper functioning of FER_GO ptest produces a quick test plot squares creates a filled-area test plot *** 5.3 Writing GO Tools *** A GO tool ("GO script", "journal file", ...) is simply a sequence of FERRET commands stored in a file and executed with the GO command. Writing a simple GO tool requires nothing more than typing normal commands into a file. To write a robust GO tool that may be shared, however, certain guidelines should be followed: 1) the GO tool should be well documented 2) the GO tool should leave the FERRET context unmodified 3) the GO tool may need to run "silently" 4) the GO tool may need to accept arguments (parameters) *** 5.3.1 Documenting GO tools *** Documentation consists primarily of well-chosen comment lines (lines beginning with an exclamation mark). In addition, a line of this form should be included: ! Description: [one-line summary of your GO tool] This line is displayed by the Fgo tool. *** 5.3.2 Preserving the FERRET context in GO tools *** Often a complex GO tool requires setting data sets, modifying the current region, etc. But to a user executing this tool its behavior may seem erratic if the user's previous context is modified by running the tool. A tool can restore the previous state of FERRET by these means: region: Save the current default region with the command DEFINE REGION/DEFAULT save. Restore it at the end of your GO tool with SET REGION save. data set: Save the current default data set with SET DATA/SAVE. Restore it at the end of your GO tool with SET DATA/RESTORE. grid: Save the current default grid set with SET GRID/SAVE. Restore it at the end of your GO tool with SET GRID/RESTORE. modes: If you modify a mode inside your GO tool by issuing a SET MODE or a CANCEL MODE command the original state of that mode can be restored using SET MODE/LAST *** 5.3.3 Silent GO tools *** If a user has set mode "verify" then by default every line of your GO tool, including comment lines, will be displayed at the screen as FERRET processes it. To make your GO tool run silently include the command CANCEL MODE VERIFY at the beginning of the GO tool and SET MODE/LAST VERIFY at the end. If the backslash character "\" is found at the beginning of any line that single line will not be displayed regardless of the state of MODE VERIFY. Thus the command "\CANCEL MODE VERIFY" is often the first line of a GO tool. Note also that the command LET/SILENT is useful in GO tools which need to define variables. *** 5.3.4 Arguments to GO tools *** Arguments (parameters) may be passed to GO tools on the command line. For example, yes? GO land red passes the string "red" into the GO file named land.jnl. Inside the GO tool the argument string "red" is substituted for the string "$1" wherever it occurs. The "1" signifies that this is the first argument - similar logic can be applied to $1,... $9 or $0 where $0 is replaced by the name of the GO tool, itself. Similarly "$*" is replaced by all the arguments, 1-9 as a single string. As FERRET performs the substitution of $1 (or other) arguments it offers a number of string processing and error processing options. For example, without these options, if a user failed to supply an argument to "GO land" then FERRET would not know what to substitute for $1 and it would have to issue an error message. A default value can be supplied by the GO tool writer using the syntax $1%string% for example, $1%black% inside land.jnl would default to "black" if no color were specified. Note that in the example percent signs were used to delimit the default string but any of the characters ! # $ % or & also work as delimiters. In another case it might not be appropriate to supply a default string but instead it would be desirable to issue an instructional error message. The "<" character indicates an error message text - $1"". Thus $1"1|red>2|" specifies a default string of "1" if no argument is given but substitutes "2" if "red" is supplied. In a typical GO tool line, defaults, options, substitutions, and an error message are combined like this: PLOT/LINE=$1"1|red>2|green>3|blue>4|2|green>3|blue>4|*>7" would never generate an error and would use line style 7 (thick black) if an unrecognized argument string such as "purple" were given. A final style note to keep in mind when writing GO tools that use arguments: providing error message feedback and appropriate documentation for the user is essential. In complex GO tools, all arguments should be checked at the beginning of the GO tool using the no-op command (has no effect) "QUERY/IGNORE". Thus the GO tool land.jnl might contain these lines at the beginning: ! check the argument QUERY/IGNORE $1"1|red|green|blue| /home/e1/tmap/fer_dsets/descr/coads_climatology.des (default) name title I J K L SST SEA SURFACE TEMPERATURE 1:180 1:90 1:1 1:12 AIRT AIR TEMPERATURE 1:180 1:90 1:1 1:12 SPEH SPECIFIC HUMIDITY 1:180 1:90 1:1 1:12 WSPD WIND SPEED 1:180 1:90 1:1 1:12 UWND ZONAL WIND 1:180 1:90 1:1 1:12 VWND MERIDIONAL WIND 1:180 1:90 1:1 1:12 SLP SEA LEVEL PRESSURE 1:180 1:90 1:1 1:12 If multiple data sets have been requested in a single FERRET session, the last requested will be the default data set. To specify other data sets, use the name of the data set or the number of the set as given by the SHOW DATA statement. For example: yes? LIST/D=2 temp will list the data for the variable "temp" in data set number 2 as displayed by SHOW DATA/BRIEF, while yes? LIST temp[D=levitus_climatology] - temp[D=coads_climatology] will list the differences between the variable "temp" in data set "levitus_climatology" and data set "coads_climatology". *** 2 NETCDF DATA *** The Network Common Data Format (NetCDF) is an interface to a library of data access routines for storing and retrieving scientific data. NetCDF allows the creation of data sets which are self-describing and platform-independent. NetCDF was created under contract with the Division of Atmospheric Sciences of the National Scientific Foundation and is available from the Unidata Program Center in Boulder, Colorado (unidata.ucar.edu). See Chapter 8 "Converting Data to NetCDF" for a complete description of how to create NetCDF data sets or how to convert existing data sets into NetCDF. To output a variable in NetCDF, simply use: yes? LIST/FORMAT=CDF variable_name LIST/FORMAT=CDF (alias SAVE) can also be used with abstract variables: yes? SAVE/FILE=example.cdf/I=1:100 sin(I/100) This will create a file named example.cdf. The current region and data sets determine the variable names in the saved file and the range over which they are saved. Saved data can then be accessed as follows: yes? USE example If a filename is not specified, FERRET will generate one. (See command SET LIST/FILE in the Commands Reference section of this manual.) An example of converting TMAP- formatted data to NetCDF goes as follows: yes? SET DATA coads_climatology yes? SAVE/L=1 sst,airt,uwnd,vwnd These commands will save sst, airt, uwnd, and vwnd at the first time step over their entire regions to a NetCDF file named by FERRET. One advantage to using NetCDF is that users on a different system (i.e. VMS instead of Unix) with different software (i.e. with an analysis tool other than FERRET) can share data easily without substantial conversion work. NetCDF files are self-describing; with a simple command the size, shape and description of all variables, grids and axes can be seen. *** 3 TMAP-FORMATTED DATA *** As of FERRET version 2.30, NetCDF is the suggested format for data storage (see Chapter 8 "Converting to NetCDF"). This section describing TMAP information is included only for users who already work with data in TMAP format. To access TMAP-formatted data sets use SET DATA_SET TMAP_set1, TMAP_set2, ... TMAP_setn must be the name of a descriptor file for a data set that is in TMAP "GT" (grids-at-timesteps) or "TS" (time series) format. ("FERRET" format and "TMAP" format are synonyms.) If the directory portion of the filename is omitted the environment variable FER_DESCR will be used to provide a list of directories to search. The order of directories in FER_DESCR determines the order of directory searches. If the extension is omitted a default of ".des" will be assumed (if the filename has more than one period, the extension must be given explicitly). Descriptors For every TMAP-formatted data set there is a descriptor file containing summary information about the contents of the data set. This includes variable names, units, grids, and coordinates. When the command SET DATA_SET is given to FERRET pointing to a GT-formatted or TS-formatted data set, it is the name of the descriptor file that must be specified. Both "GT" and "TS" formats are supported by FERRET version 2.20 and above. Special note on TMAP internal data sets: If the data set specified contains Philander/Seigel/MOM/Pacific model output, special information may be provided to FERRET to compute diagnostic variables. Normally, this information is pulled from the TMAP model runs data base ($FER_MODEL_RUNS on Unix systems), but special cases may be handled by specifying the descriptor NAMELIST parameter D_ADD_PARM. As of FERRET 2.20 D_ADD_PARM may contain: 'HEAT FLUX = DOUBLE RAMP' (or "SINGLE RAMP" or "PHILANDER/SIEGEL") 'MINIMUM WIND = 488' (or 385 or any other value) 'AIR-SEA DELTA T = 1.0' (or any other value or "LEVITUS" or "CAC") 'Am_FACTOR = 2.0 (or any other value) or for air temperature climatology data sets, only: '205_AIRT = LEV' (or "CAC","ALV"...) 'NTS30 WIND STRESS' *** 4 BINARY DATA *** Binary data words must all begin on 4-byte boundaries to be accessible by FERRET. To understand how to access binary data files with FERRET it is necessary to understand something of the structure of binary files. Binary files are of two general types - files containing record length information and files without imbedded record length information. Within each of these categories the records may in actuality be of uniform length or of variable length. *** 4.1 FORTRAN-structured binary files *** Files containing record length information are created by FORTRAN programs using the ACCESS="SEQUENTIAL" (the FORTRAN default) mode of file creation and also by FERRET using LIST/FORMAT=unf. Suppose "rrrr" represents 4 bytes of record length information and "dddd" represents a 4-byte data value. Then FORTRAN-structured files are organized in one of the following two ways: *** 4.1.1 Records of uniform length *** A FORTRAN-structured file with records of uniform length (3 single-precision floating point data values per record in this figure) look as follows: rrrr dddd dddd dddd rrrr rrrr dddd dddd dddd rrrr ... FORTRAN code that creates a data file of this type might look something like this (sequential access is the default and need not be specified in the OPEN statement): REAL VARI(10), VAR2(10), VAR3(10) ... OPEN(UNIT=20,FORMAT="UNFORMATTED",ACCESS="SEQUENTIAL",FILE="MYFILE.DAT") ... DO 10 I=1,10 WRITE (20) VAR1(I), VAR2(I), VAR3(I) 10 CONTINUE .... To access data from this file, use yes? SET DATA/EZ/FORMAT=UNF/VAR=var1,var2,var3/COL=3 myfile.dat or, yes? FILE/FORMAT=UNF/VAR=var1,var2,var3/COLUMNS=3 myfile.dat This is very similar to accessing ASCII data with the addition of the /FORMAT=unf qualifier. The /COLUMNS= qualifier tells FERRET the number of data values per record. Although optional in the above example, this qualifier is required if the number of data values per record is greater than the number of variables being read (examples follow in section "ASCII Data"). *** 4.1.2 Records of non-uniform length *** A FORTRAN-structured file with variable-length records may look as follows: rrrr dddd dddd rrrr rrrr dddd rrrr rrrr dddd dddd dddd dddd rrrr etc. With care, it is possible to read a data file containing variable-length records which was created using the simplest unformatted FORTRAN OPEN statement and a single WRITE statement for each variable. Use /FORMAT=stream to read such files. Note that sequential access is the FORTRAN default and does not need to be specified in the OPEN statement: REAL VAR1(1000), VAR2(500) ... OPEN (UNIT=20, FORMAT="UNFORMATTED", FILE="MYFILE.DAT") ... WRITE (20) VAR1 WRITE (20) VAR2 .... Use the qualifier /SKIP to skip past the record length information (/SKIP arguments are in units of words), and define a grid which will not read past the data values. The /COLUMNS= qualifier can be used when reading multiple variables to specify the number of words separating the start of each variable: yes? DEFINE AXIS/X=1:500:1 xaxis yes? DEFINE GRID/X=XAXIS mygrid yes? FILE/FORMAT=stream/SKIP=1003/GRID=mygrid/VAR=var2 myfile.dat The argument 1003 is the sum of the 1000 data words in record 1, plus 2 words of record length information surrounding the data values in record 1 (variable var1), plus 1 word of record information preceding the data in record 2. *** 4.2 Unstructured binary files *** Files without imbedded record length information are created by FORTRAN programs using ACCESS="DIRECT" in OPEN statements and by C programs. Suppose "dddd" represents a 4-byte data value. Then unstructured binary files are simply: dddd dddd dddd dddd dddd dddd ... Such files are also referred to as "direct access binary" files. The structure of the records is implied by the program accessing the data. FORTRAN code which generates a direct access binary file might look as follows: REAL*4 MYVAR(10,5) ... C Use RECL=40 for machines that specify in bytes OPEN(UNIT=20, FILE="myfile.dat", ACCESS="DIRECT", RECL=10) ... DO 100 j = 1, 5 100 WRITE (20,REC=j) (MYVAR(i,j),i=1,10) . . . . Use the following FERRET commands to read variable "myvar" from this file: yes? DEFINE AXIS/X=1:10:1 x10 yes? DEFINE AXIS/Y=1:5:1 y5 yes? DEFINE GRID/X=x10/Y=y5 g10x5 yes? FILE/VAR=MYVAR/GRID=g10x5/FORMAT=stream myfile.dat *** 5 ASCII DATA *** To access ASCII data files sets use yes? SET DATA/EZ ASCII_file_name or equivalently yes? FILE ASCII_file_name The following are qualifiers to SET DATA/EZ or FILE: Qualifier Description /VARIABLES names the variables in the file /TITLE associates a title with the data set /GRID indicates multi-dimensional data and units /COLUMNS tells how many data values are in each record /FORMAT specifies the format of the file /SKIP skips initial records of the file /ORDER specifies order of axes (which varies fastest) Use command SET VARIABLE to individually customize the variables. *** 5.1 Reading ASCII files *** Below are several examples of reading ASCII data properly. (Uniform record length, FORTRAN-structured binary data are read similarly with the addition of the qualifier /FORMAT= "unf". See Chapter 2 section "Binary Data" for other binary types). First, we look briefly at the relationship between FERRET and standard matrix notation. Linear algebra uses established conventions in matrix notation. In a matrix A(i,j), the first index denotes a (horizontal) row and the second denotes a (vertical) column. A11 A12 A13 ... A1n A21 A22 A23 ... A2n Matrix A(i,j) ... Am1 Am2 Am3 ... Amn X-Y graphs follow established conventions as well, which are that X is the horizontal axis (and in a geographical context, the longitude axis) and increases to the right, and Y is the vertical axis (latitude) and increases upward (FERRET provides the /DEPTH qualifier to explicitly designate axes where the vertical axis convention is reversed).. In FERRET, the first index of a matrix, i, is associated with the first index of an (x,y) pair - x. Likewise, j corresponds to y. Element Am2, for example, corresponds graphically to x=m and y=2. By default, FERRET stores data in the same manner as FORTRAN - the first index varies fastest. Use the qualifier /ORDER to alter this behavior. The following examples demonstrate how FERRET handles matrices. Example 1 - 1 variable, 1 dimension 1a) Consider a data set containing the height of a plant at regular time intervals, listed in a single column: 2.3 3.1 4.5 5.6 . . . To access, name, and plot this variable properly, use the commands yes? FILE/VAR=height plant.dat yes? PLOT height 1b) Now consider the same data, except listed in four columns: 2.3 3.1 4.5 5.6 5.7 5.9 6.1 7.2 . . . Because there are more values per record (4) than variables (1), use: yes? FILE/VAR=height/COLUMNS=4 plant4.dat yes? PLOT height Example 2 - 2 variables, 1 dimension 2a) Consider a data set containing the height of a plant and the amount of water given to the plant, measured at regular time intervals: 2.3 20.4 3.1 31.2 4.5 15.7 5.6 17.3 . . . To read and plot this data use yes? FILE/VAR="height,water" plant_wat.dat yes? PLOT height,water 2b) The number of columns need be specified only if the number of columns exceeds the number of variables. If the data are in six columns 2.3 20.4 3.1 31.2 4.5 15.7 5.6 17.3 ... use yes? FILE/VAR="height,water"/COLUMNS=6 plant_wat6.dat yes? PLOT height,water Example 3 - 1 variable, 2 dimensions 3a) Consider a different situation: a greenhouse with three rows of four plants and a file with a single column of data representing the height of each plant at a single time (successive values represent plants in a row of the greenhouse): 3.1 2.6 5.4 4.6 3.5 6.1 . . . If we want to produce a contour plot of height as a function of position in the greenhouse, axes will have to be defined: yes? DEFINE AXIS/X=1:4:1 xplants yes? DEFINE AXIS/Y=1:3:1 yplants yes? DEFINE GRID/X=xplants/Y=yplants gplants yes? FILE/VAR=height/GRID=gplants greenhouse_plants.dat yes? CONTOUR height When reading data the first index, x, varies fastest. Schematically, the data will be assigned as follows: x=1 x=2 x=3 x=4 y=1 3.1 2.6 5.4 4.6 y=2 3.5 6.1 . . . y=3 . . . 3b) If the file in the above example has, instead, 4 values per record: 3.1 2.6 5.4 4.6 3.5 6.1 . . . then add /COLUMNS=4 to the FILE command: yes? FILE/VAR=height/COLUMNS=4/GRID=gplants greenhouse_plants.dat Example 4 - 2 variables, 2 dimensions Like Example 3, consider a greenhouse with three rows of four plants each and a data set with the height of each plant and the length of its longest leaf: 3.1 0.54 2.6 0.37 5.4 0.66 4.6 0.71 3.5 0.14 6.1 0.95 . . . . Again, axes and a grid must be defined: yes? DEFINE AXIS/X=1:4:1 xht_leaf yes? DEFINE AXIS/Y=1:3:1 Yht_leaf yes? DEFINE GRID/X=xht_leaf/Y=yht_leaf ght_leaf yes? FILE/VAR="height,leaf"/GRID=ght_leaf greenhouse_ht_lf.dat yes? SHADE height yes? CONTOUR/OVER leaf The above commands create a color-shaded plot of height in the greenhouse, and overlay a contour plot of leaf length. Schematically, the data will be assigned as follows: x=1 x=2 x=3 x=4 ht , lf ht , lf y=1 3.1, 0.54 2.6, 0.37 5.4, 0.66 4.6, 0.71 y=2 3.5, 0.14 6.1, 0.95 . . . y=3 . . . Example 5 - 2 variables, 3 dimensions (time series) Consider the same greenhouse with height and leaf length data taken at twelve different times. The following commands will create a three dimensional grid and a plot of the height and leaf length versus time for a specific plant. yes? DEFINE AXIS/X=1:4:1 xplnt_tm yes? DEFINE AXIS/Y=1:3:1 yplnt_tm yes? DEFINE AXIS/T=1:12:1 tplnt_tm yes? DEFINE GRID/X=xplnt_tm/Y=yplnt_tm/T=tplnt_tm gplant2 yes? FILE/VAR="height,leaf"/GRID=gplant2 green_time.dat yes? PLOT/X=3/Y=2 height, leaf Example 6 - 1 variable, 3 dimensions, permuted order (vertical profile) Consider a collection of oceanographic measurements made to a depth of 1000 meters. Suppose that the data file contains only a single variable, salt. Each record contains a vertical profile (11 values) of a particular x,y (long,lat) position. Supposing that successive records are successive longistudes, the data file would look as follows (assume the equivalencies are not in the file): z=0 z=10 z=20 . . . x=30W,y=5S 35.89 35.90 35.93 35.97 36.02 36.05 35.96 35.40 35.13 34.89 34.72 x=29W,y=5S 35.89 35.91 35.94 35.97 36.01 36.04 35.94 35.39 35.13 34.90 34.72 . . . Use the qualifier /DEPTH= when defining the Z axis to indicate positive downward, and /ORDER when setting the data set to properly read in the permuted data: yes? DEFINE AXIS/X=30W:25W:1/UNIT=degrees salx yes? DEFINE AXIS/Y=5S:5N:1/UNIT=degrees saly yes? DEFINE AXIS/Z=0:1000:100/UNIT=meters/DEPTH salz yes? DEFINE GRID/X=salx/Y=saly/Z=salz salgrid yes? FILE/ORDER=zxy/GRID=salgrid/VAR=sal/COL=11 sal.dat *** Chapter 3: VARIABLES AND EXPRESSIONS *** *** 1 VARIABLES *** Variables are of 3 kinds: 1) file variables (read from disk files) 2) user-defined variables (defined by the user with LET command) 3) diagnostic variables (for GFDL/MOM Pacific ocean model runs) All 3 types may be accessed identically in all commands and expressions. All variables, regardless of kind, possess the following associated information: 1) grid - the underlying coordinate structure 2) units 3) title 4) title modifier (additional explanation of variable) 5) flag value for missing data points Use the commands SHOW DATA, SHOW VARIABLES and SHOW VARIABLES/ DIAGNOSTIC to examine the available variables of each type, respectively. The pseudo-variables I, J, K, L, X, Y, Z, T and others may be used to refer to the underlying grid locations and characteristics and to create abstract variables. *** 1.1 File variables *** File variables are stored in disk files. Input data files can be ASCII, binary, NetCDF or TMAP-formatted (see Chapter 2, Data Sets). File variables are made available with the SET DATA command. *** 1.2 Diagnostic variables *** Diagnostic variables are available only with GDFL/MOM/Philander model output. They are computed interactively by FERRET from state variables and appear as part of the data set. *** 1.3 Pseudo-variables *** Pseudo-variables are variables whose values are coordinates or coordinate information from a grid. Valid pseudo-variables are X - x axis coordinates I - x axis subscripts XBOX - size of x grid box Y - y axis coordinates J - y axis subscripts YBOX - size of y grid box Z - z axis coordinates K - z axis subscripts ZBOX - size of z grid box T - t axis coordinates L - t axis subscripts TBOX - size of t grid box A grid box is a concept needed for some transformations along an axis; it is the length along an axis that belongs to a single grid point and functions as a weighting factor during integrations and averaging transformations. The pseudo-variables I,J,K and L are subscripts; that is, they are a coordinate system for referring to grid locations in which the points along an axis are regarded as integers from 1 to the number of points on the axis. This is clear if you look at one of the sample data sets: yes? SET DATA levitus_climatology yes? SHOW DATA/FULL 1> /home/e1/tmap/fer_dsets/descr/levitus_climatology.des (default) Levitus annual climatology (1x1 degree) diagnostic variables: NOT available name title I J K L TEMP TEMPERATURE 1:360 1:180 1:20 1:1 ... on grid GLEVITR1 X=20E:20E(380) Y=90S:90N Z=0m:5000m SALT SALINITY 1:360 1:180 1:20 1:1 ... on grid GLEVITR1 X=20E:20E(380) Y=90S:90N Z=0m:5000m time-independent data file: levitus_climatology.001 We see that there are 20 points along the z-axis (1:20 under K), for example, and that the z-axis coordinate values range from 0 meters to 5000 meters. Pseudo-variables depend only on the underlying grid, and not on the variables (in this case, temperature and salt). Examples: Pseudo-variables 1) yes? LIST/I=1:10 I 2) yes? LET xflux = u * vbox[G=u] *** 1.4 User-defined variables *** New variables can be defined from existing variables and from abstract mathematical quantities (such as COS(latitude)) with command DEFINE VARIABLE (alias LET). See command DEFINE VARIABLE and command LET in the Commands Reference. Examples: User-defined variables 1) yes? LET/TITLE="Surface Relief x1000 (meters)" r1000=rose/1000 2) yes? LET/TITLE="Temperature Deviation" tdev=temp - temp[Z=@ave] *** 1.5 Abstract variables *** FERRET can be used to manipulate abstract mathematical quantities such as SIN(x) or EXP(k*t) - quantities which are independent of file variable values. Such quantities are referred to as abstract expressions. Example: Abstract variables Contour the function COS(a*Y)/EXP(b*T) where a=0.25 and b=-0.02 over the range Y=1:45 (degrees) and T=1:100 (hours) with a resolution of 1 degree on the Y axis and 2 hours on the T axis. Quick and dirty solution: yes? CONTOUR/Y=1:45/T=1:50 COS(0.25*Y)/EXP(-.02*T) Nicer (Fig. 7; plot is documented with correct units and titles): yes? DEFINE AXIS/Y=-1:45:1 /UNIT=DEGREES yax yes? DEFINE AXIS/T=1:100:2 /UNIT=HOURS tax yes? DEFINE GRID/T=tax/Y=yax my_grid yes? SET GRID my_grid yes? LET a=0.25 yes? LET b=-0.02 yes? CONTOUR/Y=-1:45/T=1:100 COS(a*Y)/EXP(b*T) See Chapter 4 section "Grids" for more information on grids. *** 2 EXPRESSIONS *** Throughout this manual, FERRET commands which require and manipulatedata are informally called "action" commands. These commands are: PLOT CONTOUR FILL (alias for CONTOUR/FILL) SHADE VECTOR WIRE LIST STAT LOAD Action commands may use any valid algebraic expression involving constants, operators (+,-,*,...), functions (SIN, MIN, INT,...), pseudo-variables (X, TBOX, ...) and other variables. A variable name may optionally be followed by square brackets containing region, transformation, data set and regridding qualifiers. For example, "temp", "salt[D=2]", "u[G=temp"], "u[Z=0:200@AVE]". The expressions may also contain a syntax of IF condition THEN expression_1 ELSE expression_2 Examples: Expressions i) temp ^ 2 temperature squared ii) temp - temp[Z=@AVE] for the range of Z in the current context, the temperature deviations from the vertical average iii) COS(Y) the cosine of the Y coordinate of the underlying grid (by default, the y-axis is implied by the other variables in the expression) iv) IF (vwnd GT vwnd[D=monthly_navy_winds]) THEN vwnd ELSE 0 use the meridional velocity from the current data set wherever it exceeds the value in data set monthly_navy_winds, zero elsewhere. *** 2.1 Operators *** Valid operators are + - * / ^ (exponentiate) AND OR GT GE LT LE EQ NE *** 2.2 Multi-dimensional expressions *** Operators and functions (discussed in the next section, Functions) may combine variables of like dimensions or differing dimensions. If the variables are of like dimension then the result of the combination is of the same dimensionality as inputs. For example, suppose there are two time series which have data on the same time axis; the result of a combination will be a time series on the same time axis. If the variables are of unlike dimensionality, then the following rules apply: 1) To combine variables together in an expression they must be "conformable" along each axis. 2) Two variables are conformable along an axis if the number of points along the axis is the same, or if one of the variables has only a single point along the axis (or, equivalently, is normal to the axis). 3) When a variable of size 1 (a single point) is combined with a variable of larger size, the variable of size 1 is "promoted" by replicating its value to the size of the other variable. 4) If variables are the same size but have different coordinates, they are conformable, but FERRET will issue a message that the coordinates on the axis are ambiguous. The result of the combination inherits the coordinates of the FIRST variable encountered that has more than a single point on the axis. Examples: Assume a region J=50/K=1/L=1 for examples 1 and 2. Further assume that variables v1 and v2 share the same x-axis. 1) yes? LET newv = v1[I=1:10] + v2[I=1:10] !same dimension (10) 2) yes? LET newv = v1[I=1:10] + v2[I=5] !newv has length of v1 (10) 3) We want to compare the salt values during the first half of the year with the values for the second half. Salt_diff will be placed on the time coordinates of the first variable - L=1:6. FERRET will issue a warning about ambiguous coordinates. yes? LET salt_diff = salt[L=1:6] - salt[L=7:12] 4) In this example the variable zero will be promoted along each axis. yes? LET zero = 0 * (i+j) yes? LIST/I=1:5/J=1:5 zero !5X5 matrix of 0's 5) Here we calculate density; salt and temp are on the same grid. This expression is an XYZ volume of points (100x100x10) of density at 10 depths based on temperature and salinity values at the top layer (K=1). yes? SET REGION/I=1:100/J=1:100 yes? LET dens = rho_un (temp[K=1], salt[K=1], Z[G=temp,K=1:10] *** 2.3 Missing value flags *** Every FERRET variable has an associated missing value flag. For ASCII (and binary) files the missing value flag is set with SET VARIABLE/BAD=. For NetCDF files it is set with the attributes missing_value and/or _FillValue. For GT and TS data it is the D_BAD_FLAG variable in the descriptor file (*.des). Abstract variables are always assigned FERRET's default flag which is -1E34 (e.g. LIST/FORMAT=(E) 1/0). In all cases in which the missing value flag is unspecified, -1E34 is the default flag. *** 2.3.1 Missing value flag in expressions *** If all of the component variables in an expression use the same bad data flag, the result inherits that bad data flag. If some of the component variables in an expression use differing bad value flags, the result inherits the bad value flag of the first variable in the expression. *** 2.3.2 Displaying the missing value flag *** If the LIST command is used, missing values are, by default, displayed as "...." To examine the flag as a numerical value, use LIST/FORMAT=(E) (or some other suitable format). *** 2.4 Functions *** Valid functions are Name #Args Description MAX 2 Compares two fields and selects the point by point maximum. MAX( temp[K=1], temp[K=2] ) returns the maximum temperature comparing the first 2 z-axis levels MIN 2 Compares two fields and selects the point by point minimum. MIN( airt[L=10], airt[L=9] ) gives the minimum air temperature comparing two timesteps INT 1 Truncates values to integers. INT( salt ) returns the integer portion of variable "salt" for all values in the current region ABS 1 Absolute value. ABS( U ) takes the absolute value of U for all points within the current region EXP 1 eX - Exponential; argument is real. EXP( X ) raises e to the power X for all points within the current region LN 1 logeX - Natural logarithm; argument is real. LN( X ) takes the natural logarithm of X for all points within the current region LOG 1 log10X - Common logarithm; argument is real. LOG( X ) takes the common logarithm of X for all points within the current region SIN 1 Trigonometric sine; argument is in radians and is treated modulo 2*pi. SIN( X ) computes the sine of X for all points within the current region COS 1 Trigonometric cosine; argument is in radians and is treated modulo 2*pi. COS( Y ) computes the cosine of Y for all points within the current region TAN 1 Trigonometric tangent; argument is in radians and is treated modulo 2*pi. TAN( theta ) computes the tangent of theta for all points within the current region ASIN 1 Trigonometric arcsine (-pi/2,pi/2). The result will be flagged as missing if the absolute value of the argument is greater than 1; result is in radians. ASIN( value ) computes the arcsine of "value" for all points within the current region ACOS 1 Trigonometric arccosine (0,pi). The result will be flagged as missing of the absolute value of the argument greater than 1; result is in radians. ACOS ( value ) computes the arccosine of "value" for all points within the current region ATAN 1 Trigonometric arctangent (-pi/2,pi/2); result is in radians. ATAN( value ) computes the arctangent of "value" for all points within the current region ATAN2 2 2-argument trigonometric arctangent of Y/X (-pi,pi); discontinuous at Y=0. ATAN2( X,Y ) computes the 2-argument arctangent of Y/X for all points within the current region MOD 2 Modulo operation ( arg1 - arg2*[arg1/arg2] ). Returns the remainder when the first argument is divided by the second. MOD( X,2 ) computes the remainder of X/2 for all points within the current region MISSING 2 Replaces missing values in the first argument (multi-dimensional variable) with the second argument; the second argument may be any conformable variable. MISSING( temp, -999 ) replaces missing values in temp with -999 MISSING( sst, temp[D=coads_climatology] ) replaces missing sst values with temperature from the COADS climatology IGNORE0 1 Replaces zeros in a variable with the missing value flag for that variable. IGNORE0( salt ) replaces zeros in salt with the missing value flag RANDU 1 Generates a grid of uniformly distributed [0,1] pseudo-random values. The first valid value in the field is used as the random number seed. Values that are flagged as bad remain flagged as bad in the random number field. RANDU( temp[I=105:135,K=1:5] ) generates a field of uniformly distributed random values of the same size and shape as the field "temp[I=105:135,K=1:5]" using temp[I=105,k=1] as the pseudo-random number seed. RANDN 1 Generates a grid of normally distributed pseudo-random values. As above, but normally distributed rather than uniformly distributed. RHO_UN 3 Calculates rho (density kg/m^3) from salt (psu), temperature (deg C) and pressure (decibars) using the 1980 UNESCO International Equation of State (IES80). The routine uses the high pressure equation of state from Millero et al. 1980 and the one-atmosphere equation of state from Millero and Poisson 1981 as reported in Gill 1982. The notation follows Millero et al. 1980 and Millero and Poisson 1981. RHO_UN( salt, temp, Z ) THETA_FO 4 Calculates local potential pressure field at salt (psu), temperature (deg C), pressure (decibars) and specified reference pressure. This calculation uses Bryden 1973 polynomial for adiabatic lapse rate and Runge-Kutta 4-th order integration algorithm. References: Bryden,H., 1973, Deep-Sea Res.,20,401-408 Fofonoff,N,M, 1977, Deep-Sea Res.,24,489-491. THETA_FO( salt, temp, Z, 0 ) *** 2.5 Transformations *** Transformations (e.g. averaging, integrating, etc.) may be specified along the axes of a variable. Some transformations (e.g. averaging) reduce a range of data to a point; others (e.g. differentiating) retain the range. When transformations are specified along more than one axis of a single variable the order of execution is X axis first, then Y then Z then T. Example syntax: TEMP[Z=0:100@LOC:20] (depth at which temp has value 20) Valid transformations are Default Transform Argument Description @DIN definite integral (weighted sum) @IIN indefinite integral (weighted running sum) @AVE average @VAR unweighted variance @MIN minimum @MAX maximum @SHF 1 pt shift @SBX 3 pt boxcar smoothed @SBN 3 pt binomial smoothed @SHN 3 pt Hanning smoothed @SPZ 3 pt Parzen smoothed @SWL 3 pt Welch smoothed @DDC centered derivative @DDF forward derivative @DDB backward derivative @NGD number of valid points @NBD number of bad (invalid) points flagged @SUM unweighted sum @RSUM running unweighted sum @FAV 3 pt fill missing values with average @FLN:n 1 pt fill missing values by linear interpolation @FNR:n 1 pt fill missing values with nearest point @LOC 0 coordinate of ... (e.g. depth of 20 degrees) @WEQ "weighted equal" (integrating kernel) The command SHOW TRANSFORM will produce a list of currently available transformations. Examples: Transformations u[Z=0:100@AVE] - average of u between 0 and 100 in Z sst[T=@SBX:10] - box-car smooths sst with a 10 time point filter tau[L=1:25@DDC] - centered time derivative of tau v[L=@IIN] - indefinite (accumulated) integral of v qflux[X=@AVE,Y=@AVE] - XY area-averaged qflux *** 2.5.1 General information about transformations *** Transformations are normally computed axis by axis; if multiple axes have transformations specified simultaneously (e.g. U[Z=@AVE,L=@SBX:10]) the transformations will be applied sequentially in the order X then Y then Z then T. There are two exceptions to this: if @DIN is applied simultaneously to both the X and Y axes (in units of degrees of longitude and latitude, respectively) the calculation will be carried out on a per-unit-area basis (as a true double integral) instead of a per-unit-length basis, sequentially. This ensures that the COSINE(latitude) factors will be applied correctly. The same applies to @AVE simultaneously on X and Y. Data that are flagged as invalid are excluded from calculations. When calculating integrals and derivatives (@IIN, @DIN, @DDC, @DDF, and @DDB) FERRET attempts to use standardized units for the grid coordinates. If the underlying axis is in a known unit of length FERRET converts grid box lengths to meters. If the underlying axis is in a known unit of time FERRET converts grid box lengths to seconds. If the underlying axis is degrees of longitude a factor of COSINE (latitude) is applied to the grid box lengths in meters. If the underlying axis units are unknown FERRET uses those unknown units for the grid box lengths. (If FERRET does not recognize the units of an axis it displays a message to that effect when the DEFINE AXIS or SET DATA command defines the axis.) See command DEFINE AXIS/UNITS in the Commands Reference in this manual for a list of recognized units. All integrations and averaging are accomplished by multiplying the width of each grid box by the value of the variable in that grid box - then summing and dividing as appropriate for the particular transformation. If integration or averaging limits are given as world coordinates, the grid boxes at the edges of the region specified are weighted according to the fraction of grid box that actually lies within the specified region. If the transformation limits are given as subscripts, the full box size of each grid point along the axis is used - including the first and last subscript given. The region information that is listed with the output reflects this. Some transformations (derivatives, shifts, smoothers) require data points from beyond the edges of the indicated region in order to perform the calculation. FERRET automatically accesses this data as needed. It flags edge points as missing values if the required beyond-edge points are unavailable (e.g., @DDC applied on the X axis at I=1). *** 2.5.2 General information about smoothing transformations *** FERRET provides several transformations for smoothing variables (removing high frequency variability). These transformations replace each value on the grid to which they are applied with a weighted average of the surrounding data values along the axis specified. For example, the expression u[T=@SPZ:3] replaces the value at each (I,J,K,L) grid point of the variable"u" with the weighted average - u at t = 0.25*(u at t-1) + 0.5*(u at t) + 0.25*(u at t+1) The various choices of smoothing transformations (@SBX, @SBN, @SPZ, @SHN, @SWL) represent different shapes of weighting functions or "windows" with which the original variable is convolved. New window functions can be obtained by nesting the simple ones provided. For example, using the definitions yes? LET ubox = u[L=@SBX:15] yes? LET utaper = ubox[L=@SHN:7] produces a 21 point window whose shape is a boxcar (constant weight) with COSINE (Hanning) tapers at each end. FERRET may be used to directly examine the shape of any smoothing window: Mathematically, the shape of the smoothing window can be recovered as a variable by convolving it with a delta function. In the example below we examine (PLOT) the shape of a 15 point Welch window (Fig. 8). ! define X axis as [-1,1] by 0.2 yes? GO unit_square yes? SET REGION/X=-1:1 yes? LET delta = IF X EQ 0 THEN 1 ELSE 0 ! convolve delta with Welch window yes? PLOT delta[I=@SWL:15] *** 2.5.3 @DIN - definite integral *** The transformation @DIN computes the definite integral - a single value that is the integral between two points along an axis (compare with @IIN). It is obtained as the sum of the grid_box*variable product at each grid point. Grid points at the ends of the indicated range are weighted by the fraction of the grid box that falls within the integration interval. If @DIN is specified simultaneously on both an X axis of longitude and a Y axis of latitude the calculation will be performed as a double integration rather than as sequential single integrations. Example: yes? CONTOUR/X=160E:160W/Y=5S:5N u[Z=0:50@DIN] *** 2.5.4 @IIN - indefinite integral *** The transformation @IIN computes the indefinite integral - at each subscript of the result it is the value of the integral from the start value to the upper edge of that grid box. It is obtained as a running sum of the grid_box*variable product at each grid point. Grid points at the ends of the indicated range are weighted by the fraction of the grid box that falls within the integration interval. Example: yes? CONTOUR/X=160E:160W/Z=0 u[Y=5S:5N@IIN] *** 2.5.5 ***@AVE - average The transformation @AVE computes the average weighted by grid box size - a single number representing the average of the variable between two endpoints. If @AVE is specified simultaneously on both an X axis of longitude and a Y axis of latitude the calculation will be performed as a double integration rather than as sequential single integrations. Example: yes? CONTOUR/X=160E:160W/Y=5S:5N u[Z=0:50@AVE] Note that the unweighted mean can be calculated using the @SUM and @NGD transformations. *** 2.5.6 @VAR - weighted variance *** The transformation @VAR computes the weighted variance of the variable with respect to the indicated region (ref. Numerical Recipes, The Art of Scientific Computing, by William H. Press et al., 1986). *** 2.5.7 ***MIN - minimum The transformation @MIN finds the minimum value of the variable within the specified axis range. Example: For fixed Z and Y yes? PLOT/T="1-JAN-1982":"1-JAN-1983" temp[X=160E:160W@MIN] plots a time series of the minimum temperature found between longitudes 160 east and 160 west. *** 2.5.8 ***@MAX - maximum The transformation @MAX finds the maximum value of the variable within the specified axis range. See also @MIN. *** 2.5.9 ***@SHF:n - shift The transformation @SHF shifts the data up or down in subscript by the number of points given as the argument. Examples: u[L=@SHF:2] associates the value of U[L=3] with the subscript L=1. u[L=@SHF:1]-U gives the forward difference of the variable U along the L axis. *** 2.5.10 ***@SBX:n - boxcar smoother The transformation @SBX applies a boxcar window (running mean) to smooth the variable along the indicated axis. The width of the boxcar is the number of points given as an argument to the transformation. All points are weighted equally, regardless of the sizes of the grid boxes, making this transformation best suited to axes with equally spaced points. If the number of points specified is even, however, @SBX weights the end points of the boxcar smoother as 1/2. Example: yes? PLOT/X=160W/Y=0 u[L=1:120@SBX:5] The transformation @SBX does not reduce the number of points along the axis; it replaces each of the original values with the average of its surrounding points. Regridding can be used to reduce the number of points. *** 2.5.11 ***@SBN:n - binomial smoother The transformation @SBN applies a binomial window to smooth the variable along the indicated axis. The width of the smoother is the number of points given as an argument to the transformation. The weights are applied without regard to the widths of the grid boxes making this transformation best suited to axes with equally spaced points. Example: yes? PLOT/X=160W/Y=0/Z=0 u[L=1:120@SBN:15] The transformation @SBN does not reduce the number of points along the axis; it replaces each of the original values with a weighted sum of its surrounding points. Regridding can be used to reduce the number of points. The argument specified with @SBN, the number of points in the smoothing window, must be an odd value; an even value would result in an effective shift of the data along its axis. *** 2.5.12 @SHN:n - Hanning smoother *** Transformation @SHN applies a Hanning window to smooth the variable along the indicated axis (ref. Numerical Recipes, The Art of Scientific Computing, by William H. Press et al., 1986). In other respects it is identical in function to the @SBN transformation. Note that the Hanning window used by FERRET contains only non-zero weight values with the window width. Some interpretations of this window function include zero weights at the end points. Use an argument of N-2 to achieve this effect (e.g., @SBX:5 is equivalent to a 7 point Hanning window which has zeros as it first and last weights). *** 2.5.13 @SPZ:n - Parzen smoother *** Transformation @SPZ applies a Parzen window to smooth the variable along the indicated axis (ref. Numerical Recipes, The Art of Scientific Computing, by William H. Press et al., 1986). In other respects it is identical in function to the @SBN transformation. *** 2.5.14 @SWL:n - Welch smoother *** Transformation @SWL applies a Welch window to smooth the variable along the indicated axis (ref. Numerical Recipes, The Art of Scientific Computing, by William H. Press et al., 1986). In other respects it is identical in function to the @SBN transformation. *** 2.5.15 ***@DDC - centered derivative The transformation @DDC computes the derivative with respect to the indicated axis using a centered differencing scheme. The units of the underlying axis are treated as they are with integrations. If the points of the axis are unequally spaced, note that the calculation used is still (Fi+1 - Fi-1) / (Xi+1 - Xi-1) . Example: yes? PLOT/X=160W/Y=0/Z=0 u[L=1:120@DDC] *** 2.5.16 ***@DDF - forward derivative The transformation @DDF computes the derivative with respect to the indicated axis. A forward differencing scheme is used. The units of the underlying axis are treated as they are with integrations. Example: yes? PLOT/X=160W/Y=0/Z=0 u[L=1:120@DDF] *** 2.5.17 @DDB - backward derivative *** The transformation @DDF computes the derivative with respect to the indicated axis. A backward differencing scheme is used. The units of the underlying axis are treated as they are with integrations. Example: yes? PLOT/X=160W/Y=0/Z=0 u[L=1:120@DDB] *** 2.5.18 @NGD - number of good points *** The transformation @NGD computes the number of good (valid) points of the variable with respect to the indicated axis. Use @NGD in combination with @SUM to determine the number of good points in a multi-dimensional region. *** 2.5.19 @NBD - number of bad points *** The transformation @NBD computes the number of bad (invalid) points of the variable with respect to the indicated axis. Use @NBD in combination with @SUM to determine the number of bad points in a multi-dimensional region. *** 2.5.20 @SUM - unweighted sum *** The transformation @SUM computes the unweighted sum (arithmetic sum) of the variable with respect to the indicated axis. This transformation is most appropriate for regions specified by subscript. If the region is specified in world coordinates, the edge points are not weighted - they are wholly included in or excluded from the calculation depending on the location of the grid points with respect to the specified limits. *** 2.5.21 @RSUM - running unweighted sum *** The transformation @RSUM computes the running unweighted sum of the variable with respect to the indicated axis. @RSUM is to @IIN as @SUM is to @DIN. The treatment of edge points is identical to @SUM. *** 2.5.22 @FAV - averaging filler *** The transformation @FAV fills holes (values flagged as invalid) in variables with the average value of the surrounding grid points along the indicated axis. The width of the averaging window is the number of points given as an argument to the transformation. All of the surrounding points are weighted equally, regardless of the sizes of the grid boxes, making this transformation best suited to axes with equally spaced points. If the number of points specified is even, however, @FAV weights the end points of the filling region by 1/2. If any of the surrounding points are invalid they are omitted from the calculation. If all of the surrounding points are invalid the hole is not filled. Example: yes? CONTOUR/X=160W:160E/Y=5S:0 u[X=@FAV:5] *** 2.5.23 @FLN:n - linear interpolation filler *** The transformation @FLN:n fills holes in variables with a linear interpolation from the nearest non-missing surrounding point. n specifies the number of points beyond the edge of the indicated axis limits to include in the search for interpolants (default n = 1). Unlike @FAV, @FLN is sensitive to unevenly spaced points and computes its linear interpolation based on the world coordinate locations of grid points. *** 2.5.24 @FNR:n - nearest neighbor filler *** The transformation @FNR:n is similar to @FLN:n, except that it replicates the nearest point to the missing value. In the case of points being equally spaced around the missing point, the mean value is used. *** 2.5.25 @LOC - location of *** The transformation @LOC accepts an argument value - the default value is zero if no argument is specified. The transformation @LOC finds the single location at which the variable first assumes the value of the argument. The result is in units of the underlying axis. Linear interpolation is used to compute locations between grid points. If the variable does not assume the value of the argument within the specified region the @LOC transformation returns an invalid data flag. For example, temp[Z=0:200@LOC:18] finds the location along the Z axis (often depth in meters) at which the variable "temp" (often temperature) first assumes the value 18, starting at Z=0 and searching to Z=200. yes? CONTOUR/X=160E:160W/Y=10S:10N temp[Z=0:200@LOC:18] produces a map of the depth of the 18 degree isotherm. See also the General Information section in this chapter. Note that the transformation @LOC can be used to locate non-constant values, too, as the following example illustrates: Example: locating non-constant values Determine the depth of maximum salinity. yes? LET max_salt = salt[Z=@MAX] yes? LET zero_at_max = salt - max_salt yes? LET depth_of_max = zero_at_max[Z=@LOC:0] *** 2.5.26 @WEQ - weighted equal; integration kernel *** The @WEQ ("weighted equal") transformation is the subtlest and arguably the most powerful transformation within FERRET. It is a generalized version of @LOC; @LOC always determines the value of the axis coordinate (the variable X, Y, Z, or T) at the points where the gridded field has a particular value. More generally, @WEQ can be used to determine the value of any variable at those points. Like @LOC, the transformation @WEQ finds the location along a given axis at which the variable is equal to the given (or default) argument. For example, V1[Z=@WEQ:5] finds the Z locations at which V1 equals "5". But whereas @LOC returns a single value (the linearly interpolated axis coordinate values at the locations of equality) @WEQ returns instead a field of the same size as the original variable. For those two grid points that immediately bracket the location of the argument, @WEQ returns interpolation coefficients. For all other points it returns missing value flags. If the value is found to lie identically on top of a grid point an interpolation coefficient of 1 is returned for that point alone. Example 1 yes? LET v1 = X/4 yes? LIST/X=1:6 v1, v1[X=@WEQ:1], v1[X=@WEQ:1.2] X v1 @WEQ:1 @WEQ:1.2 - ----- ----- ----- 1: 0.250 .... .... 2: 0.500 .... .... 3: 0.750 .... .... 4: 1.000 1.000 0.2000 5: 1.250 .... 0.8000 6: 1.500 .... .... The resulting field can be used as an "integrating kernel", a weighting function that when multiplied by another field and integrated will give the value of that new field at the desired location. Example 2 Using variable v1 from the previous example, suppose we wish to know the value of the function X^2 (X squared) at the location where variable v1 has the value 1.2. We can determine it as follows: yes? LET x_squared = X^2 yes? LET integrand = x_squared * v1[X=@WEQ:1.2] yes? LIST/X=1:6 integrand[X=@SUM] !FERRET output below X_SQUARED * V1[X=@WEQ:1.2] X: 1 to 6 (summed) 23.20 Notice that 23.20 = 0.8 * (5^2) + 0.2 * (4^2) Below are two "real world" examples which produce fully labelled plots. Example 3: salinity on an isotherm Use the Levitus climatology to contour the salinity of the Pacific Ocean along the 20-degree isotherm (Fig. 9). yes? SET DATA levitus_climatology ! annual sub-surface climatology yes? SET REGION/X=100E:50W/Y=45S:45N ! Pacific Ocean yes? LET isotherm_20 = temp[Z=@WEQ:20] ! depth kernel for 20 degrees yes? LET integrand_20 = salt * isotherm_20 yes? SET VARIABLE/TITLE="Salinity on the 20 degree isotherm" integrand_20 yes? PPL CONSET .12 !contour label size (def. .08) yes? CONTOUR/LEV=(33,37,.2) integrand_20[Z=@SUM] yes? GO fland !continental fill Example 4: month with warmest sea surface temperatures Use the COADS data set to determine the month in which the SST is warmest across the Pacific Ocean. In this example we use the same principles as above to create an integrating kernel on the time axis. Using this kernel we determine the value of the time step index (which is also the month number, 1-12) at the time of maximum SST (Fig. 10). yes? SET DATA coads_climatology ! monthly surface climatology yes? SET REGION/X=100E:50W/Y=45S:45N ! Pacific Ocean yes? SET MODE CAL:MONTH yes? LET zero_at_warmest = sst - sst[l=@max] yes? LET integrand = L[G=sst] * zero_at_warmest[L=@WEQ] ! "L" is 1 to 12 yes? SET VARIABLE/TITLE="Month of warmest SST" integrand yes? SHADE/L=1:12/PAL=inverse_grayscale integrand[L=@SUM] *** Chapter 4: GRIDS AND REGIONS *** *** 1 OVERVIEW *** Information describing a region in space/time, a data set, and a grid is collectively referred to as the "context". The current context may be examined with the commands SHOW DATA_SET, SHOW REGION and SHOW GRID. The context may be set explicitly with the commands SET DATA_SET, SET REGION and SET GRID. The context may be modified for the duration of a single command with qualifiers to the command name (separated by slashes). The same qualifiers in square brackets may also modify single variables, changing the context only of that variable: yes? PLOT/D=levitus_climatology temp, salt yes? CONTOUR rose[D=etopo20] yes? FILL/Z=0 temp[L=2] - temp[L=1] *** 2 GRIDS *** Every variable has an underlying grid which defines a coordinate space. All grids are in a sense 4 dimensional (X,Y,Z and T) but axes normal to the data are represented as "normal" (such as the Z axis of the surface wind stress). Grids can be viewed, specified and created using SHOW GRID, SET GRID, DEFINE AXIS, and DEFINE GRID. These commands are all in the Commands Reference section of this manual. Data can be regridded by the G= modifier. (See Chapter 4 section "Regridding".) *** 2.1 Defining Grids *** Axes and grids can be created by DEFINE AXIS and DEFINE GRID. NetCDF and TMAP-formatted data set variables have all of the necessary grid and axis definitions embedded in the data set files, but if you are reading data from an ASCII or binary file, you must tell FERRET about the underlying grid of your data. If you are creating abstract expressions entirely from pseudo-variables, you may want to define a grid in order to define the coordinate space of your calculation. This will also help produce a nicely labelled plot. (See example in Chapter 3 section "Abstract Variables".) Example This example is taken from the demonstration script "file_reading_demo.jnl". An ASCII file contains a grid of numbers, 50 rows by 6 columns. Suppose the data are on a 2D grid of 6 longitudes by 50 latitudes (Fig. 11). yes? DEFINE AXIS/X=10E:60E:10/UNIT=DEGREE xlong yes? DEFINE AXIS/Y=0:49N:1/UNIT=DEGREE ylat yes? DEFINE GRID/X=xlong/Y=ylat gsnoopy2d ! By default only 1 column is read, /COLUMNS= specifies 6 columns yes? FILE/VAR=my_2D_var/COL=6/GRID=gsnoopy2d snoopy.dat yes? CONTOUR my_2D_var *** 2.2 Regridding *** Syntax: var[G=name] for (default) linear interpolation to new grid or var[G=name@trn] to regrid all axes using transform "trn" (see below) or var[G=name,GX=@TRN,GY=@TRN,...] to control regridding transformations along each axis separately where var is the name of the variable (e.g. temp, u, tau, ...) name is the name of a variable (e.g. temp[G=u]) or the name of a grid (e.g. temp[G=gu01]) trn is the name of a special transformation (e.g. @AVE, @ASN, @LIN) The FERRET distribution provides a demonstration of regridding: yes? GO regridding_demo Regridding is essential for algebraic operations that combine variables on incompatible grids. FERRET provides the commands DEFINE AXIS and DEFINE GRID to assist with the creation of arbitrary grids. FERRET insists on consistent dimensionality during regridding operations. It is not permissible for a variable that is normal to a given axis to be directly regridded to a grid that has defined locations along that axis or visa versa. Again, type "GO regridding_demo" at the FERRET prompt for some graphical examples. Examples 1) Suppose the variables u and temp are on staggered X, Y, and Z axes but share the same T axis. Then the two variables can be multiplied together on the axes (grid) of the u variable as follows: yes? CONTOUR u * temp[G=u] This will regrid temp onto the u grid by multi-axis linear interpolation before performing the multiplication. 2) Two variables, v1 and v2, are defined on distinct 4-dimensional grids (X,Y,Z, and T axes). The T axes of the two grids are identical but the X, Y, and Z axes all differ between the two variables. (This is often the case in numerical model outputs.) To obtain the variable v1 on its original Z (depth) locations but regridded in the XY plane to the grid locations of the variable v2, define a new grid (say, named "new_grid") that has the X and Y axes of v2 but the Z axis of v1. yes? DEFINE GRID/LIKE=v2/Z=v1 new_grid !define new grid yes? LIST/X=160E:140W/Y=5S:5N v1[G=new_grid] !request regridding 3) In this example we look at temperature data from two data sets. levitus_climatology, an annual climatology, has the variable "temp" on an XYZ grid which is 1x1 degree in XY, and coads_climatology, a monthly climatology, has the variable "sst" on an XYT grid which is 2x2 degrees in XY. Suppose we wish to look at the sea surface temperatures in January at the higher XY resolution of the Levitus data. yes? SET DATA levitus_climatology yes? SET DATA coads_climatology yes? SET REGION/L=1/Z=0 yes? !get the name of the grid on which temp is defined yes? SHOW GRID temp[D=levitus_climatology] ! --> "glevitr1" yes? DEFINE GRID/X=glevitr1/Y=glevitr1/Z=sst/L=sst glevitus_xy yes? LIST/X=150E:155E/Y=0:5N sst[G=glevitus_xy] *** 2.2.1 Regridding transformations *** FERRET version 3.1 supports three regridding transformations. @LIN, the default, performs regridding by multi-axis linear interpolation. @AVE computes the length-weighted average of all points on the source grid that lie partly or completely within each grid box of the destination grid. @ASN associates by subscript (blindly) the points from the source grid onto destination coordinates (grid sites normally match). Note: When @AVE is applied simultaneously to several axes, the default is to treat each axis as an independent averaging operation. If, however, the axes are X and Y, and X and Y are longitude and latitude, respectively, then an area-weighted average (weighted by cos(latitude)) is used. Examples 1) Let variable temp be defined on a grid with points spaced regularly at 1 degree intervals in both longitude and latitude (X and Y). Let grid "g10" possess points spaced regularly at 10 degree intervals in both X and Y. yes? PLOT temp[G=g10] ! uses linear interpolation (@LIN) yes? PLOT temp[G=g10@AVE] !area-averages 10x10 degrees of source\ points into each destination point. yes? PLOT temp[G=g10,GX=@AVE] !averages 10 degrees of longitude but\ interpolates (@LIN) in Y. 2) @ASN has the effect of bypassing FERRET's protections against misrepresenting data (Fig. 12). yes? SET DATA levitus_climatology yes? SET REGION/X=180/Y=0 ! true profile yes? PLOT/Z=0:5000 temp yes? DEFINE AXIS/DEPTH/Z=100:2000:100 zfalse yes? DEFINE GRID/LIKE=temp/Z=zfalse gfalse ! false profile yes? PLOT/Z=0:5000/OVER temp[G=gfalse@ASN] *** 3 REGIONS *** The region in space and time where expressions are evaluated may be specified in 3 different ways: 1) with the command SET REGION 2) with qualifiers to the command name (slash-delimited) 3) with qualifiers to variable names (in square brackets, comma-delimited) If SET REGION is used, FERRET remembers the region as the default context for future commands, whereas a qualifier to a command name specifies the region for that command only, and a qualifier to a variable name specifies the region for that variable and command only. Regions may be manipulated using DEFINE REGION, SET REGION, @ notation and CANCEL REGION. The Commands Reference section of this manual covers all of these topics. Region information is normally specified in the following form: QUAL=val or QUAL=lo_val:hi_val or QUAL=val@transform (as a variable qualifier only) or QUAL=lo_val:hi_val@transform (as a variable qualifier only) When the region for an axis is specified as a single value (instead of a range) FERRET, by default, selects the grid point of the grid box containing this value. The FERRET mode "interpolate" can control this behavior. See command SET MODE INTERPOLATE in Commands Reference. Examples: Regions Examples of valid region specifications. 1) Fully specify the region in an XY plane with the first vertical (Z) level and time 27739. yes? SET REGION/X=140E:160W/Y=10S:20N/K=1/T=27739 2) Contour vertical heat advection within whatever region is the current default (previously set with SET REGION). yes? CONTOUR qadz 3) Define, modify and set a named region and then modify with delta notation. yes? DEFINE/REGION/Y=5S:5N YT !define region YT to be 5S:5N yes? DEFINE REGION/DY=-1:+1 YT !modify region YT to be 6S:6N yes? SET REGION/@YT !set current region to YT yes? SET REGION/DY=-1:+1 !modify current region to 7S:7N 4) List meridional currents calculated by averaging values between the surface and a depth of 50 m. yes? LIST v[Z=0:50@AVE] 5) Equivalent to v[Z=10] - v[Z=0:100@AVE], the anomaly at z=10 between v and the 0 to 100 meter depth average of v. yes? LIST/Z=10 v - v[Z=0:100@AVE] *** 3.1 Latitude *** Specify latitude or a latitude range with the qualifier Y or J. Specifications using J are integers between 1 and the number of points on the Y axis. Specifications using Y are in the units of the Y axis. The units may be examined with SHOW GRID/Y. If the Y axis units are degrees of latitude then the Y positions may be specified as numbers followed by the letters "N" or "S". Examples yes? CONTOUR temp[Y=15S:10N] yes? LIST/J=50 u *** 3.2 Longitude *** Specify longitude or a longitude range with the qualifier X or I. Specifications using I are integers between 1 and the number of points on the X axis. Specifications using X are in the units of the X axis. The units may be examined with SHOW GRID/X. If the units are degrees then X values may be given as numbers followed by "W" or "E" (e.g.160E,110.5W) or as values between 0 and 360 with Greenwich at 0 increasing eastward. Note: If the X axis is "modulo" then it is sometimes desirable to use X greater than 360. Examples yes? CONTOUR temp[Y=160E:140W] yes? LIST/I=100 u yes? SHADE/X=100:460 temp !360 degrees centered at 100W See Chapter 4 section "Modulo Axes" for help with globe-encircling axes. *** 3.3 Depth *** Specify depth or a depth range with the qualifier Z or K. Specifications using K are integers between 1 and the number of points on the Z axis. Specifications using Z are in the units of the Z axis. The units may be examined with SHOW GRID/Z. Examples yes? CONTOUR temp[Z=0:100] yes? LIST/K=3 u *** 3.4 Time *** Specify time or a time range with the qualifier T or L. Specifications using L are integers between 1 and the number of points on the T axis. Specifications using T may refer to calendar dates or to the time step units in which the time axis of the data set is defined. Calendar date/time values are entered in the format dd-mmm-yyyy:hh:mm:ss, for example 14-FEB-1988:12:30:00. At a minimum the string must contain day, month and year. If the string contains any colons it must be enclosed in quotation marks to differentiate from colons used to designate a range. If a time increment is specified with the repeat command given in calendar format (e.g. REPEAT/T="1-JAN-1982":"15-JAN-1982":6) it is interpreted as hours always. Calendar dates in the years 0000 and 0001 are regarded as year-independent dates (suitable for climatological data). SHOW GRID/T can be used to display time step values. (Units may vary between data sets.) The commands SET MODE CALENDAR and CANCEL MODE CALENDAR can be used to view date strings or time steps, respectively. Examples yes? LIST/T="1-JAN-1982:13:50":"15-FEB-1982" density yes? CONTOUR temp[T=27740:30000] yes? LIST/L=90 u See Chapter 4 section "Modulo Axes" for help with climatological axes. *** 3.5 Delta *** The notation q=lo:hi:delta (e.g. Y=20S:20N:5) specifies that the data in the requested range is regularly subsampled at interval "delta". This notation is valid only for the REPEAT, SHOW GRID and DEFINE AXIS commands, and the qualifiers /XLIMITS and /YLIMITS used in action commands with graphical output. *** 3.6 @ notation *** Regions may be named and referred to using the syntax "@name". Some commonly used regions are predefined. See commands SET REGION and DEFINE REGION in the Commands Reference section for further information. If a region is specified using a combination of "@" notation and explicit axis limits the explicit axis limits will be evaluated after the "@" specification, possibly superseding the "@" limits. Using the @ notation only sets/alters the axis limits specified in the named region. For example, suppose that region "XY" is defined for the X and Y axes, but not for the Z and T axes. Then yes? SET REGION/@XY modifies only X and Y limits. BUT, yes? SET REGION XY modifies all axes - X and Y to the limits specified by XY, and Z and T to unspecified (even if they were previously specified). Examples 1) Contour the 25th time step of temperature data at depth 10 within region T, the "Tropical Pacific". yes? CONTOUR/@T/Z=10/L=25 temp 2) Produce a contour plot over region W, the "Whole Pacific Ocean", in the XY plane (the variable to be contoured as well as the depth and time will be inferred from the current context). yes? CONTOUR/@W var1 3) Set the default region to "T", the Tropical Pacific Ocean (latitude 23.5S to 23.5N). yes? SET REGION/@T 4) Define a region and then supersede with an axis limit specification. yes? DEFINE REGION/X=180:140W/Y=2S:2N/Z=5 BOX1 yes? SET REGION/@BOX1/Z=15 !replace Z Pre-defined regions As a convenience in the analysis of TMAP's model outputs over the Tropical Pacific Ocean the following regions are pre-defined: Name Region Latitude Longitude T Tropical Pacific 23.5S:23.5N 130E:70W N Narrow Pacific 10.0S:10.0N 130E:70W W Whole Pacific 30.0S:50.0N 130E:70W These may be redefined by the user for the duration of a FERRET session or until the definitions are canceled. *** 3.7 Modulo axes *** Some axes are inherently "modulo" indicating that the axis wraps around - the first point immediately following the last. To determine if an axis is modulo use SHOW AXIS or SHOW GRID. A letter "m" following the number of points in the axis indicates a modulo axis. The command SHOW GRID qualified by the appropriate axis limits can be used to examine any part of the axis -including points beyond the nominal length of the axis. The commands SET AXIS/MODULO and CANCEL AXIS/MODULO can be used to control this feature on an axis-by-axis basis. Example yes? SET DATA coads_climatology yes? SHOW GRID/I=180:183 sst !range request beyond last point GRID COADS1 name axis # pts start end COADSX LONGITUDE 180mr 21E 19E(379) [text omitted] I X BOX_SIZ 180> 19E(379) 2 181> 21E(381) 2 182> 23E(383) 2 183> 25E(385) 2 The most common uses of modulo axes are: 1) As longitude axes for globe-encircling data sets. This allows any starting and any ending longitudes to be used, for example, X=140E:140E indicates the entire earth with data beginning and ending at 140E. 2) As time axes for climatological data. By this device the time axis appears to extend from 0 to infinity and the climatological data can be referred to at any point in time. This facilitates comparisons with data sets at fixed times. As of FERRET version 3.1 a restriction exists on the use of modulo axes. For GT- formatted and NetCDF data sets only a single axis can be accessed modulo style at a time. Thus in a 12 month, globe-encircling climatology, it is valid to request X=180/L=1:36 or X=180:540/L=5, but not X=180:540/L=1:36. *** Chapter 5: ANIMATIONS *** *** 1 OVERVIEW *** A sequence of FERRET plots can be stored and then animated. Each plot is stored as one frame in a movie file. FERRET stores movie frames in Hierarchical Data Format (HDF), a format designed by the National Center for Supercomputing Applications (NCSA). A movie file can then be displayed as an animated sequence of frames with NCSA's xds - X Data Slice (not distributed with FERRET; see Chapter 4 section "Displaying a movie", for details). *** 2 CREATING A MOVIE *** Creating a movie requires two steps: 1) designate an output file with SET MOVIE 2) generate a sequence of frames with REPEAT and FRAME See commands SET MOVIE, CANCEL MOVIE, SHOW MOVIE, FRAME and REPEAT in the Commands Reference section of this manual. Example: basic movie yes? SET MOVIE/FILE=my_movie.mgm !specify output file yes? SET DATA coads_climatology !specify data set yes? SET REGION/@W !specify Pacific Ocean yes? LET/TITLE="SST Anomaly" SST_ANOM = SST - SST[L=1:12@AVE] yes? REPEAT/L=1:12 FILL/FRAME SST_ANOM !filled contour of sea \ surface temp anomaly \ captured and written to HDF \ file yes? CANCEL MOVIE !close movie file (not required) If you omit the SET MOVIE/FILE= command, the default name "ferret.mgm" will be assigned to the movie file. REPEAT executes its argument (in the above example, FILL) successively for each timestep specified. REPEAT can have multiple arguments separated by semi-colons and enclosed in parentheses. FRAME is a stand-alone command, but also a qualifier for the graphical output commands PLOT, CONTOUR, FILL (alias for CONTOUR/FILL), SHADE, VECTOR and WIRE. The saved animation frames are exactly the size and shape of the window from which they are created. Thus a large window results in a larger, slower animation that demands more disk space and memory to play back. The SET WINDOW/SIZE= command is generally used to specify minimally acceptable frame size. See Chapter 4 section "Advanced Moviemaking" for more examples. *** 3 DISPLAYING A MOVIE *** Viewing a movie requires software which is not included with the FERRET distribution (although in some cases we have made the binary available in FERRET's anonymous ftp area). NCSA's X Data Slice reads HDF files and is available via anonymous ftp from NCSA. It requires about 1.7Mb of disk space. NCSA's ftp server is ftp.ncsa.uiuc.edu login id is "anonymous", give your email address as the password Consult the README files you will find there for instructions on obtaining X Data Slice. Other utilities from NCSA can also be used for animations. *** 4 ADVANCED MOVIE-MAKING *** *** 4.1 REPEAT command *** The REPEAT command is quite flexible - it allows you to repeat a sequence of commands, not just a single command as in the basic example above. You can give the GO command as an argument to REPEAT. The following examples demonstrate these techniques. Note: MODE VERIFY must be SET (this is the default state) for loop counting to work. Example 1 Here we give multiple arguments to REPEAT; note the semi-colon separation and the parentheses. Note that FRAME, in this is example, is used as a stand-alone command. yes? REPEAT/L=1:12 (FILL SST; GO fland; FRAME) Example 2 In this example the user calls setup_movie.jnl (text included below), title.jnl, which creates a title frame, then repeats main_movie.jnl (text included below) for each time step desired. Finally, the user adds a frame of credits at the end of the movie. Each of the scripts would end with the FRAME command (except setup_movie). Using GO scripts as arguments to REPEAT allows you customize the plot with many commands before finally issuing FRAME, as the text of main_movie.jnl below demonstrates. yes? ! make the movie yes? GO setup_movie yes? GO title yes? REPEAT/L=1:12 GO main_movie yes? GO credits ! setup_movie.jnl SET WINDOW/SIZE=.45/ASPECT=0.7 SET MOVIE/file=my_movie.mgm SET DATA coads_climatology SET REGION/X=130E:75W/Y=8S:8N SET MODE CALENDAR:months GO bold PPL SHAKEY ,,.15,.2 PPL AXLEN 8.8,4.8 ! main_movie.jnl FILL/SET_UP/LEVELS=(16,31,1) sst PPL LABS; PPL TITLE PPL FILL LABEL 210,9.5,0,0,.22 @TRCOADS MONTHLY CLIMATOLOGY (1946-1989) LABEL 210,-12,0,0,.22 @TRSEA SURFACE TEMPERATURE (DEG C) LABEL 130,11,-1,0,.22 @TR'LAB4' FRAME Note: If you use the FILL command, we suggest that you use SHADE while customizing and fine-tuning your movie, then use FILL for the final run. SHADE is much faster. *** 4.2 Initializing the color table *** If you create a movie with a title frame, or a first frame which otherwise uses different colors than the rest of the movie, you should be aware of an HDF peculiarity: all the colors which you plan to use in your movie must be in the first frame, or else color behavior will be unpredictable when you animate. To "reserve" the colors you need, use overlapping full-window viewports. Make a representative plot in the title frame, then cover over it with either a black or white rectangle and finally write the title text. Here is a script which initializes the color table while creating a title frame. ! define 3 identical full-frame viewports DEFINE VIEW full1; DEFINE VIEW full2; DEFINE VIEW full3 ! draw frame one of the movie in full color SET VIEW full1 SET DATA coads_climatology SHADE/LEVELS=(16,31,1)/L=1 sst ! dummy frame ! white-out over the picture SET VIEW full2 GO setup_text SHADE/PALETTE=white/NOLAB/NOKEY/i=1:2/j=1:2 (i+j)*0 !put on title frame labels (using [0,1] coordinate space) SET VIEW full3 GO setup_text PPL PLOT LABEL .5,.7,0,0,.3 @TRMy Title PPL ALINE 1,.2,.55,.8,.55 PPL ALINE 1,.2,.53,.8,.53 LABEL .5,.4,0,0,.2 @CRBy me !capture the title frame and clean up FRAME GO cleanup_text *** Chapter 6: CUSTOMIZING PLOTS *** *** 1 OVERVIEW *** Detailed control is possible over most aspects of FERRET graphical outputs. A custom modification will require the user to either add a qualifier to a FERRET command or communicate directly with the graphical package PPLUS, which is contained inside of FERRET. The most commonly used PPLUS commands are listed in the following sections of this chapter. Consult the PLOT PLUS for FERRET manual for complete command lists and the specifics of command syntax. Ferret communicates with PPLUS by sending a sequence of commands to PPLUS (the command PPL ECHO ON causes the sequence of commands that FERRET sends to PPLUS to be logged in the file fort.40.). The user can give further commands to PPLUS directly using the FERRET command PPL (e.g. yes? PPL AXLEN 10,7). Some results can be attained in two ways - with either FERRET or PPLUS commands. However, the interaction of the two is complex and the inexperienced user may get unexpected results, so when possible, use only FERRET commands. PPLUS uses a deferred mode of output - various commands are given to PPLUS which describe the plot state but produce no immediate output; the entire plot is then rendered by a single command. Some plot states (e.g. axis labels) are set by FERRET with every plotted output; to customize these states it is necessary to use the /SET_UP qualifier (which sets up the plot inside of PPLUS) and then modifiy the state with direct PPL commands. Other plot states are never set by FERRET and if modified at any time remain in their specified state for all subsequent plots. Still other states are modified by FERRET only under special circumstances. Here is a very simple customization (Fig. 13): yes? PLOT/X=1:100/TITLE="My SIN Plot"/SET_UP sin(x/6) !use /SET_UP yes? PPL YLAB "SIN value" yes? PPL PLOT The examples throughout this chapter show how the /SET_UP qualifier on graphics commands can be used to delay rendering of a plot while the user modifies plot appearance with PPLUS commands. Below is a list of PPLUS commands which are reset by FERRET: PPLUS command when reset by FERRET XFOR, YFOR reset for every plot XLAB, YLAB reset for every plot XAXIS, YAXIS reset for every plot LABS reset for every plot ALINE reset for every plot TAXIS OFF reset for every plot TITLE reset for every plot TICS reset for every plot (small tic size, only) WINDOW ON reset for every plot PEN 1,n reset for every plot LIMITS reset for every plot ORIGIN reset by SET WINDOW/ASPECT and SET VIEWPORT; Y origin may be shifted to accommodate many line style keys AXLEN modified by SET WINDOW/ASPECT and SET VIEWPORT VIEWPORT modified by WIRE/VIEW LEV modified by CONTOUR and SHADE unless /LEVELS_SAME given VECSET modified by VECTOR unless /LENGTH_SAME given WINDOW modified for "fresh" plots but not for overlay plots *** 2 GRAPHICAL OUTPUT *** *** 2.1 FERRET graphical output controls *** FERRET command Function CONTOUR produces a contour plot of a single field FILL alias for CONTOUR/FILL;produces color-filled contour plot PLOT produces a line or symbol plot of one or more arrays SHADE produces a shaded representation (rectangular cells) VECTOR produces a vector arrow plot WIRE produces a 3D wire frame plot SET WINDOW manipulates graphics windows SET VIEWPORT places graphics output into a sub-window (pane) *** 2.2 PPLUS graphical output commands *** Whenever a plot is customized using /SET_UP to delay display, the plot will ultimately be rendered using a PPLUS graphical output command (not the FERRET counterpart). A customized contour or filled-contour plot is rendered with PPL CONTOUR, a wire frame plot with PPL VIEW and so on. Command Function CONTOUR makes a contour plot PLOT plots x-y pairs for all lines of data PLOTUV makes a stick plot of vector data SHADE makes a shaded representation VIEW makes a wire frame plot VECTOR makes a plot of a vector field The graphical output command PLOTUV can be used to make stick plots easily, as the following time series example shows. yes? SET DATA coads; SET REGION/X=180/Y=0/L=400:500 yes? PLOT/SET uwnd, vwnd yes? PPL PLOTUV *** 3 AXES *** By default, FERRET displays X- and Y-axes with tics and numeric labels at reasonable intervals and a label for each axis. Time axes are also automatically formatted and used as needed. These axis features can be modified or suppressed using the following FERRET direct controls and PPLUS commands. *** 3.1 FERRET axis controls *** The following qualifiers are used with graphical output commands PLOT, VECTOR, SHADE, and CONTOUR to specify axis limits, tic spacing and possible axis reversal: FERRET qualifers /XLIMITS, /YLIMITS The /XLIMITS and /YLIMITS qualifiers use the syntax /XLIMITS=lo:hi:delta. Tic marks are placed every "delta" units, starting at "lo" and ending at "hi". Every other tic mark is labelled. "delta" may be negative, in which case the axis is reversed. The following arguments to SET MODE and CANCEL MODE determine axis style (e.g. SET MODE CALENDAR:days) : FERRET arguments CALENDAR LATIT_LABEL LONG_LABEL See the Commands Reference section of this manual for more information. *** 3.2 PPLUS axis commands *** Command Function XAXIS* controls numeric labeling and tics on the X axis (redundant with /XLIMITS) YAXIS* controls numeric labeling and tics on the Y axis (redundant with /YLIMITS) AXATIC sets number of large tics automatically for X and Y AXLABP locates or omits axis labels at top/bottom or left/right of plot AXLEN** sets axis lengths AXLINT sets numeric label interval for axes every nth large tic AXLSZE sets axis label heights AXNMTC sets number of small tics between large tics on axes AXNSIG sets no. significant digits in numeric axis labels AXSET allows omission of plotting of any axis AXTYPE sets axis type (linear, log, inv. log) for x- and y-axis TICS sets axis tic size and placement inside or outside axes XFOR* sets format of x-axis numeric labels YFOR* sets format of y-axis numeric labels XLAB* sets label of x-axis YLAB* sets label of y-axis TXLABP establishes time axis label position (or absence) TXTYPE* sets the style of the time axis TXLINT* specifies which time axis tics will be labeled TXLSZE sets height of time axis labels TXNMTC sets number of small tics between large tics on time axis * issued by FERRET with every relevent plot ** issued by FERRET upon SET WINDOW/ASPECT or SET VIEWPORT Examples 1) Plot with no axis labels (character or numeric) and no tics (Fig. 14). (Equivalent to - yes? GO box_plot PLOT/I=1:10/NOLABEL 1/i ) yes? PLOT/i=1:30/NOLABEL/SET 1/i yes? PPL AXLABP 0,0 !turn off numeric labels yes? PPL TICS 0,0,0,0 !suppress small and large tics yes? PPL PLOT !render plot yes? PPL TICS .125,.25,.125,.25 !reset tics to default yes? PPL AXLABP -1,-1 !reset numeric labels 2) customize x axis label (Fig. 15; XLAB always reset by FERRET) yes? PLOT/SET/i=1:100 sin(x/6) yes? PPL XLAB My Custom Axis Label yes? PPL PLOT 3) specify tic frequency for y axis yes? PLOT/i=1:30/YLIM=0:1:.2 1/i *** 4 LABELS *** FERRET, by default, produces labelled axes, a plot title, documentation about the plot axes normal to the plot, and a signature (current date and FERRET version number) when a plot is rendered. The /NOLABELS qualifier suppresses the plot title, the documentation and signature, but not the axis labels; PPLUS commands XLAB,YLAB and AXLABP control axis labels. *** 4.1 Adding labels *** The FERRET command LABEL adds a label to a plot and takes the following arguments: yes? LABEL xpos,ypos,center,angle,size text where xpos and ypos are in user (axis) units, size is in inches, angle is in degrees (0 at 3 o'clock) and center is -1, 0, or +1 for left, center, or right justification. The label position will adjust itself automatically when the plot aspect ratio or the viewport is changed. If you prefer to locate labels using inches rather than using data units issue the command yes? LABEL/NOUSER xpos,ypos,... Note, however, that the layout of a plot in inches - lengths of axes, label positions, etc. - shifts with changes in window aspect ratio (SET WINDOW/ASPECT) and with the use of viewports. Labels specified using LABEL/NOUSER will need to be adjusted if the aspect ratio or viewport is changed. Notes: 1) If you use the command PPL LABS instead of LABEL, be aware that when defining a new movable label, all lower-numbered labels must already be defined. 2) The FERRET command LABEL is an alias for PPL %LABEL. PPLUS does NOT consider a label created with LABEL a movable label. Consequently, no label number is assigned and the label cannot be manipulated as a movable label. 3) %LABEL is an unusual command in that the label appears on the plot immediately after the command is given, rather than being deferred. This has ramifications for the user who has multiple plot windows open and is in MODE METAFILE, since a metafile is not closed until a new plot is begun. If the user produces a plot in window B, and then returns to a previous window A and adds a label with LABEL, that label will appear on the screen correctly, but will be in the metafile corresponding to window B. Example yes? PLOT/I=1:100 sin(i/6) yes? LABEL 50, 1.2, 0, 0, .2 @P2MY SIN PLOT *** 4.2 Removing movable labels *** Removing a movable label is a two step process: identifying the label number and then deleting the label. PPLUS internally refers to all movable labels with label reference numbers. The PPLUS command LIST LABELS will list the PPLUS labels and the text strings they contain. Then the user can use 'GO unlabel n', where n is the reference number, to delete a label. Example In this example we plot the same figure in two viewports, one plot with the default "signature", and one plot with the signature removed (Fig. 16). !upper viewport has a "signature" yes? PPL BOX on yes? SET VIEW upper yes? PLOT/I=1:100 sin(i/6) !in the lower viewport !the signature has been removed yes? SET VIEW lower yes? GO unlabel 1 yes? GO unlabel 2 yes? GO unlabel 3 yes? PPL PLOT yes? CANCEL VIEWPORT *** 4.3 FERRET label controls *** In addition to LABEL (discussed above), FERRET controls include the /NOLABELS qualifier, which suppresses default plot title, documentation and signature, and /TITLE qualifier to graphical output commands PLOT, SHADE, CONTOUR, VECTOR and WIRE: FERRET qualifiers /NOLABELS /TITLE= and arguments to SET MODE and CANCEL MODE: FERRET arguments ASCII_FONT CALENDAR LATIT_LABEL LONG_LABEL *** 4.4 PPLUS label commands *** FERRET stores the text strings of the following labels in PPLUS symbols. The symbol names are: symbol name label LABTIT title label LABX X axis label LABY Y axis label LABn nth movable label As stated above, PPLUS commands regarding movable labels are largely superceded by the FERRET command LABEL and 'GO unlabel n'. Command Function LABS* makes, removes or alters a movable label HLABS sets height of each movable label RLABS sets angle for each movable label LABSET sets character heights for labels LLABS sets start position for and draws a line to a movable label TITLE* sets and clears main plot label XLAB* sets label of X axis YLAB* sets label of Y axis * issued by FERRET with every relevent plot Example This example customizes a plot using PPLUS label controls. yes? PLOT/Z=20/I=1:100/SET_UP z * sin(i/6) yes? PPL LABS 4,48,0,0 @p2'lab4' yes? PPL HLABS 4,.25 yes? PPL LABS/NOUSER 5,0,6.3,-1 *** Magnified SIN function *** yes? PPL LABSET ,,,.35 yes? PPL PLOT *** 5 COLOR *** FERRET and PPLUS use colors stored by index. Storage indices 0 and 1 are used as window background and foreground colors. Indices 1-6 are reserved for lines. As the user makes SHADE and FILL requests, each color is assigned to the next available storage index beginning at 7, and that assignment is automatically "protected" when viewports or color overlays are added. If your SHADE and FILL commands request more colors than there are storage indices(256), you will be informed with an error message and the color behavior may become unpredictable. For example, if you have multiple viewports defined within a window you may run out of color storage indices. If you are using the same color palette(s) in each viewport, yo