Hello

I wonder if someone on the list can give me some guidelines on how to use Grid:-Seq.

I have applied Grid:-Set and Grid:-Seq to one of my problems where parallelization is a possibility.   In this problem, I have a massive list of lists that needs to be processed in chunks and then a new list is returned.

What I've found so far:

1) Grid-Seq won't work if I use a procedure from a private module that calls local (or exported) functions of the same module.  Solution:  I need to use savelib (there is a bug in Grid:-Seq)

2) Grid-Seqs seems to be working fine up to a point where it simply stops.  For example:   Starting with 431895 sets, the procedure divides them into 864 sets of 500 elements (the last one does not usually contain 500 elements).  Then I use res:=Grid:-Seq['tasksize'=9](myFunction(newSets[i],...,....),i=1..864).   The messages sent out by Grid:-Seq are 

Seq: using tasksize=9; number of elements= 864, number of partitions = 96

......

Seq: done sending all partitions

Seq: received results from all nodes; collating results

After two hours running and using 36 cores, the result is available in res.  The next step is to use the results in res for another cycle of calculations.  1503462 sets = 3007 sets of 500 elements.  The messages sent out by Grid:Seq are 

Seq: using tasksize=9; number of elements= 3007, number of partitions = 335

......

Seq: done sending all partitions

The last message showed up on the screen after an hour. During the process, I saw the current tasks finishing and new ones starting.  However I could not see "Seq: received results from all nodes; collating result" after a day running.  I am using timelimit to be sure that all calculations will finish no matter what.  

I have checked the system information and found that: 1) all 72 cores are running but they are jumping from 0% to 100% (in the example that works I could see all of them in 100% solid), 2) Memory is at 156 GB (200 Gb is the limit) and no swap to disk.

I have also noticed:

1. If I use Maple 2017, not even the first example works.
2. tasksize=9 somehow helps with similar problems.  If the size is left for Maple to decide, it seems that the problem happens for sets of smaller size.
3. On linux I had to use "kill -9" to remove maple from every single core. My impression is that Grid:-Seq does not clean up the processes after they finished.  
4.  The problem seems to be on "results from all nodes; collating result".

I know that the explanation above in rather vague, but if someone has any clue or guidelines on how to solve this problem, please share it with me.

Many thanks

Ed

Hello

Unfortunately I run out of ideas on how to solve the problem of searching and removing elements in a variable size list.  Below you can find an example.

parms:=[seq(seq(alpha[i,j],j=0..9),i=1..3)];
abc:=combinat:-choose(parms,3):

the condition to remove the element of the list is given by the following procedure.

conds:= proc(varCoef::list,vars::name:=alpha)
local T1,T2,res;
#
T1 := table([2=3,3=2,5=6,6=5,7=9,9=7]):
T2 := table([2=3,3=2]):
#
res:=map(u->vars[`if`(assigned(T2[op(1,u)]),T2[op(1,u)],op(1,u)),
         `if`(assigned(T1[op(2,u)]),T1[op(2,u)],op(2,u))],
    varCoef);
res:=convert(convert(res,set),list):
end proc:

Something like

member(conds(abc[1]),abc,'k')

if it is a member, the element indexed by k should be removed from abc and then the search continues using the next element in abc (that now has one less element).   Otherwise, the element is kept and the search moves to next one.  The process ends when the last element in a short version of abc is tested.

Thanks all for the help and patience.

Ed

Hello

I need to test a huge amount of subsets and for doing that I was thinking of using combinat:-subset, since it avoids allocating memory for all subsets.  The problem with subset for my specific application is that it generates sets with all possible sizes and I just need subsets with a chosen number of elements.  Can that be done in maple?  

Many thanks

Ed

Hello

Moving from Mathematica to Maple and back these couple of days is driving me insane.  I get stuck trying to translate very simple things for not knowing each command belongs to each software.  Therefore I do apologize for another silly question. 

Given the list of indexed variables

varA := [A[1, 0], A[1, 1], A[1, 2], A[1, 3], A[1, 4], A[1, 5], A[1, 6], A[1, 7], A[1, 8], A[1, 9], A[2, 0], A[2, 1], A[2, 2], A[2, 3], A[2, 4], A[2, 5], A[2, 6], A[2, 7], A[2, 8], A[2, 9], A[3, 0], A[3, 1], A[3, 2], A[3, 3], A[3, 4], A[3, 5], A[3, 6], A[3, 7], A[3, 8], A[3, 9]]

how to apply the following substitution 

varA/. {Subscript[A, m_, 2] -> Subscript[B, m, 3], 
  Subscript[A, m_, 3] -> Subscript[B, m, 2], 
  Subscript[A, m_, 5] -> Subscript[B, m, 6], 
  Subscript[A, m_, 6] -> Subscript[B, m, 5], 
  Subscript[A, m_, 7] -> Subscript[B, m, 9], 
  Subscript[A, m_, 9] -> Subscript[B, m, 7]}

After trying a couple of commands such as map, subs, etc. I decide to try fromMma but no avail.  

Many thanks for the patience and help.

Ed

Hello

I have lots of lists of indexed variables (I am not sure if that is the correct name for it. Please correct me if it is not).  One example is (length of the list can change as well as the indices of the variables):

ans1 := [alpha[3, 7], alpha[3, 8], alpha[3, 9]]

and I need to retrieve two lists from it:  1) [3,3,3] and 2) [7,8,9].  I could not figure out how to do it.

Many thanks.

Ed