One of the goals, possibly the main goal, of this web book, High Performance Maple Programming Techniques, is to demonstrate methods for increasing the speed of Maple computations. To determine whether a particular algorithm is faster than another, the execution time of the two must be accurately measured. Speed, however, is not the sole criterion for judging performance. Memory usage is also a significant metric, particularly if the algorithm must work with large data sets. This chapter describes a method for measuring the execution times and memory usage of competing Maple procedures.

The primary difficulty in accurately measuring the performance of several procedures is ensuring that the Maple enviroment remains constant for each test run. The best way to achieve that is to start a new Maple session for each measurement; that, however, makes writing the test code and comparing the results a nuisance.

The next best approach is to issue a restart command between each test run. That causes Maple to clear its internal memory and return to the operating system most of the memory that it has allocated. This can be done in the Maple worksheet environment, however, because the code required to initialize the data must be duplicated for each procedure that is tested, this method can be tedious and unwieldy. For example, changing the initial data requires editing the same code in several places in the worksheet. Because each restart command clears internal memory, collecting the results into a single table so that they can be quickly compared requires writing the results of each test to a file and then reading them back into the worksheet. Finally, while the worksheet enviroment is excellent for interacting with Maple and for presenting Maple results, a programmer's text editor (emacs, vi, jedit, etc) is better for writing Maple code.

The following sections describe a method that uses tty maple (command-line maple) to generate a table comparing the execution time and memory usage of alternative procedures. All code being tested may be stored in a single text file. The initialization code that is executed need only be entered in one place, so it can be readily modified. To accomplish this, Maple preprocessor macros are used to recreate the initialization data between restart commands. The use of Maple preprocessor macros precludes using the Maple worksheet interface for the tests.

      #-------- Define Preprocesser Macros ----------
      $define LOOPS 200 # number of time procedure is executed
      $define NUM 50    # number of words in list
      $define LENGTH 5  # number of characters in each word
      $define ARGS [seq](StringTools:-Random(LENGTH, 'alpha'), cnt=1..NUM)

      $define MEAS(i) \
        kernelopts('gcfreq'=10^7): \
        StringTools:-Randomize(1): \
        Usage:-TimeMemoryPrint(SortNames[i], argseq = ARGS, loops = LOOPS ): \
        restart;

      #-------- Print Notes and Table Header -----------
      printf("Testfile: %s\n\n", TESTFILE):
      printf("Compare performance of sorting names.\n"):
      printf("Each test sorts the same list of %d names of %d characters %d times.\n\n", NUM, LENGTH, LOOPS):
      Usage:-TimeMemoryHeader();

      restart;

      #------- Assign and Measure each Procedure -------

      SortNames[2] := proc(names :: list)
          sort( names
                , (a,b) -> sprintf("%a",a) <= sprintf("%a",b)
              );
      end proc:

      MEAS(2);

      SortNames[4] := proc(names :: list)
      local n;
          map(attributes, sort([seq](setattribute([sprintf]("%a",n),n)
                                     , n = names)
                               , 'lexorder'[1]));
      end proc:

      MEAS(4);

      done
    

      Testfile: sort-names.mpl

      Compare performance of sorting names.
      Each test sorts the same list of 50 names of 5 characters 200 times.

      time (s)  used (MB)  alloc (MB)  gctimes  proc
      --------  ---------  ----------  -------  ----
        1.43     18.89       14.37         1    SortNames[2]
        0.17      2.51        1.50         1    SortNames[4]
    

The LOOPS, NUM, and LENGTH macros correspond, respectively, to the number of repetitions the test procedure is executed, the number of names in each list, and the length of each name.

The ARGS macro generates the list of names, using the defined macros for parameters. The resulting list is regenerated and passed to each test procedure during measurements.

The MEAS macro has one parameter, the index of the test. The macro expands to a series of Maple statements that are executed for each measurement. The following describes the effect and purpose of each statement.

• The call to kernelopts('gcfreq'=10^7) reduces the occurrence of garbage collection.
• StringTools:-Randomize(1) ensures that the same list of names is generated for each test.
• Usage:-TimeMemoryPrint(...) passes the expression sequence contained in the list [ARGS] to SortNames[i] and executes it LOOPS times. It computes the time and memory usage and prints a single row in the output table.
• The restart command restarts Maple. This ensures that each test starts with the same environment.

The first printf statement makes use of the preprocessor macro TESTFILE, which is assigned by tmaple (see below) to print the name of the file being tested The next two printf statements print the common conditions for the test. The call to Usage:-TimeMemoryHeader() prints the table header and the row of underscores beneath it. A restart command before the next part ensures that each test seems almost the same starting condtions.

Each procedure to be measured (Sort[2] and Sort[4]) is assigned and then measured using the previously defined macro MEAS. The Usage package is a small, custom package that exports commands for measuring and tabulating the time and memory usage of procedures being tested. The source code, Usage.mpl, and compiled Maple archive, Usage.mla, are attached at the bottom of this web page. The package exports three procedures:

• TimeMemoryGC(ex :: uneval, loops :: posint := 1)
  Evaluates the expression ex in a loop loops times and returns a sequence of four expressions:
  1. the duration (in seconds) of the loops to evaluate
  2. the additional memory used (in bytes)
  3. the additional memory allocated (in bytes)
  4. the number of garbage collections.
• TimeMemoryPrint(p, {argseq := NULL, loops :: posint := 1} )
  Calls TimeMemoryGC( p(argseq), loops ) and prints the result as a row of a table. The arguments to the procedure and the number of loops are keyword parameters (a new feature as of Maple 10); their defaults are NULL and 1, respectively. It returns NULL.
• TimeMemoryHeader
  Prints a header for the table. It returns NULL.

After installing the Usage.mla archive in a directory that is in your libname path, you can measure the performance of, for example, the test file sort-names.mpl by issuing the command

      $ maple -q sort-names.mpl
      

While that works, a possible disadvantage to calling tty maple directly is that it normally loads the user's Maple initialization file, which may assign additional, unneeded procedures. A convenient alternative is to execute a shell script that calls tty maple using specific options to avoid reading the initialization file. The attached file tmaple does just that. Note that this is a Linux shell script; it should be usable in Windows via cygwin.

To use the shell script, install it in your path and do

      $ tmaple sort-names.mpl
      

• tmaple.txt This file should be renamed tmaple, the txt extension was necessary to upload it to the MaplePrimes site.
• sort-names.mpl Sample test file.
• Usage.mpl Source code for Usage.mla.
• Usage.mla Maple archive.