dohashi

1192 Reputation

10 Badges

19 years, 77 days
I am a Senior Software Developer in the Kernel Group, working on the Maple language interpreter. I have been working at Maplesoft since 2001 on many aspects of the Kernel, however recently I have been focusing on enabling parallel programming in Maple. I have added various parallel programming tools to Maple, and have been trying to teaching parallel programming techniques to Maple programmers. I have a Master's degree in Mathematics (although really Computer Science) from the University of Waterloo. My research focused on Algorithm and Data Structure, Design and Analysis.

MaplePrimes Activity


These are answers submitted by dohashi

I took a quick look at the code you posted and I have a few suggestions.

First, parallel programming is more complex than sequential, and many of the guidelines for writing good sequential code become even more important in parallel.  Use procedure parameters and return values, not global variables.  This is good practice in general, but in parallel, global variables are almost always a bad idea.  Things like r and W should be passed in as parameters not used as globals, even if they are read only.

It looks like your sequential PR and parallel calls are returning different values.  This suggests that they are not performing the same task.  There is no point worrying about performance before correctness.

In addition, the problem size seems somewhat small.  Maybe you intend to run this on a large problem when it is working, but for now, the input may be to small to see meaningful gains from parallelism.

Although the load and unload options still work, they have been superceeded by the ModuleLoad and ModuleUnload functions.  Its basically a cleaner implementation of the same thing.

The reason your load function is not being called is that the load function is only called when the module is loaded from a repository.  This doesn't happen when you define the module by reading the file into Maple.  You need to save the module to a repository and then load the module from the library to have the load function be called.

If you are just trying to test the module, you can call the load function directly, right after the module has been defined.

When running Maple via the OpenMaple interface, Maple is running as a library in another application.  That application (The Java app in the case of JOpenMaple) has control over the stacklimit.  If you want to increase the stack limit for Maple, you need to increase it for the main application.  As Maple is not in control of the real stack limit, it can be dangerious to adjust the stack limit in OpenMaple. In fact, we should probably disable it when running in open maple.

Darin

ps.  The following should help you set the java stack size:

https://stackoverflow.com/questions/3700459/how-to-increase-the-java-stack-size

-- Kernel Developer Maplesoft

I though this was an interesting question and I wanted to provide a really good solution so I created a blog post about it:

http://www.mapleprimes.com/posts/153046-Using-Tasks-To-Filter-Combinations

My solution parallelizes the generation of combinations as well as the filtering.  I hope you find it helpful.

Darin

-- Kernel Developer Maplesoft

Maple uses a JDBC driver to connect to the database.  The driver provides a cursor like interface to Maple, however the jdbc driver decides how to buffer data from the server.  Make sure you have the most recent version of the jdbc driver available for your database as newer drivers may make smarter decisions.  In particular, it looks like recent versions of the MS SQL jdbc driver has better handling of buffering than the older versions.  You could also try specifying the selectMethod=direct option for the MS SQL jdbc driver.  With selectMethod=direct, the jdbc driver should pull down all the data at once.  However this can be bad if you also want to work with large results as well.

This post gives some more information on this

http://dbaspot.com/ms-sqlserver/223585-selectmethod-cursor-selectmethod-direct.html

I hope this helps.

Darin

Try

writedata( fg, [p] );

or use fprintf.  fprintf gives you more control over the format of the output.

fprintf( fg, "%f\n", p );

Darin

-- Kernel Developer Maplesoft

1.  This is how you initialize the newly created object.  See the ?ModuleCopy help page.

2.  Maple allows objects to override existing global routines, so you could write an object that has its own "sin" routine.  That routine will be called when the object is passed as an argument to the global routine.

> o := module()
>     option object;
>
>     export sin::static;
>
>     sin := proc() print( "method override" ); end;
> end;
                              o := Object<<47288987569152>>

>
> sin( o );
                                    "method override"

 

See ?object,methods for more information

Darin

-- Kernel Developer Maplesoft

I wrote a series of blogs on parallal programming in Maple:

http://www.mapleprimes.com/users/dohashi/posts

That page lists them from newest to oldest, so go to the end of the list to see the first posts.

One comment I'll make is that these posts assume the reader is familar with Maple programming in general, so you may want to take a look that the Programming Guide if you need to learn basic Maple programming (writing procedures for example)

Darin

-- Kernel Developer Maplesoft

Assuming the calculation for result1 is independant of result2, it would probably be best to use two loops, one for result1 and one for result2.  Something like this should work:

> p1 := proc(N)
>      local i, result;
>
>      for i from 1 to N
>      do
>          result[i] := 2*i-1
>      end;
>
>      op(result);
> end:
>
> p2 := proc(N)
>      local i, result;
>
>      for i from 1 to N
>      do
>          result[i] := 2*i;
>      end;
>
>      op(result);
> end:
>
> Threads:-Task:-Start( passed, Task=[p1,10], Task=[p2,10] );
table([1 = 1, 2 = 3, 3 = 5, 4 = 7, 5 = 9, 6 = 11, 7 = 13, 9 = 17, 8 = 15, 10 = 19]),

    table([1 = 2, 2 = 4, 3 = 6, 4 = 8, 5 = 10, 6 = 12, 7 = 14, 9 = 18, 8 = 16, 10 = 20])

If the calculation of result[i] is independant of the calculation of result[j], then you might be able to get more parallelism by parallelizing the for loop itself.

Darin

-- Kernel Developer Maplesoft

Sounds like a problem with lots of repeated calculations.  Can you figure out a way to structure the mulitplications so you can reuse intermediate results?

See https://secure.wikimedia.org/wikipedia/en/wiki/Dynamic_programming

Darin

-- Kernel Developer Maplesoft

It is worth pointing out that the most of Maple's packages, including the Sockets package, have not been verified as thread safe, therefore they may not work correctly in parallel.  Because we have not verified Sockets as thread safe and because it is implemented as an external library, Maple linearizes calls into the sockets external library.  Therefore code like this will not show a significant speed up in parallel.

You can see my post on the limitation of parallelism in Maple for more information.

Darin

-- Kernel Developer Maplesoft

You will get this exception when your operating system refuses to give Maple more memory.  There are a few different scenarios when this can happen.  The most common one is that Maple is consuming all the memory on your machine.  Eventually you will run out of memory.  Another possibility (and this is more likely on Mac/Unix than it is on windows) is that the operating system is imposing a memory limit on the Maple kernel, and Maple has used all that memory.  Finally, it is possible that Maple is trying to allocate a single huge object, say 10^(10^10). 

You can see how much memory Maple thinks it has access to but calling kernelopts( datalimit ).

> kernelopts( datalimit );
                                         1482000

The return value is in kilobytes (well kibibytes technically).

If the return value is "infinity", then Maple thinks it has no limits (except the total memory available on the system).

If the limit is much lower than the total amount of RAM on your machine, then you probably have a OS imposed memory limit.  One of the comments on the post that pagan linked to describes how to fix that on the Mac.

Darin

-- Kernel Developer Maplesoft

I'm having a difficult time understanding exactly what you want.

Do you want to have an interactive prompt while this code is executing?  If so, then what you want is not possible.  Having the interactive prompt blocks the kernel from sending updates to the interface.  This is a limitation of the communication channel between the kernel and the interface.  The only thing I can think of would be to write your output to a file and then watch the contents of the file.

Now, if you don't need the interactive prompt then you should be able to do what you want by having one background thread running a loop with a delay and checking some global variables, and the main thread doing the real work (and, of course, updating the global variables so the other thread can see them).

Darin

-- Kernel Developer Maplesoft

On further reflection, what you want won't work.

Although the kernel can happily run more that one thread at a time, the interface between the kernel and the GUI cannot handle asynchronious updates.  So while the main thread is waiting for user input (aka waiting for you to enter some code and hit enter). the background thread is not able to send an update to the GUI.

If you were to execute some code from the prompt, you would probably see some updates make it through, but once the control returned to the user, the background thread would be blocked again.

Sorry.

Darin

-- Kernel Developer Maplesoft

 

Here is an example that shows a background thread running as you requested, but instead of updating a math container, it just prints to the screen:

thread_done := false;

f := proc()
    local st;
    global thread_done;

    while not thread_done
    do
        st := time[real](); 
        print( st );
        while time[real]()-st < 1 
        do  
        end do;
    end do;
end;

thread_id := Threads:-Create( f() );

st := time[real](); 
while time[real]()-st < 20 do  end do ;

thread_done := true;
Threads:-Wait( thread_id );

You can do whatever you like betwen the Threads:-Create() and Threads:-Wait(), and the thread will keep executing.

The reason examples like this are not in the help pages, is that we are mostly focused on using parallelism to speed up computation, rather than allowing "background processing".

Darin

-- Kernel Developer Maplesoft

1 2 Page 1 of 2