Hi Georgios, I can't seem to reproduce the problem with Browse, on my 11.02 in 64bit Linux, sorry. But I can answer your second question. The setting interface(rtablesize) controls the cutoff size below which one only sees that "summary" detail of the Vector/Matrix/Array. acer

I just realized that the commands in the string could all be written out more nicely, in one execution group, as say, H := " sin(x); int(cos(x),x); plot(x^2,x=1..2); x^3; ": If the string is then split at the symbol "\n", then it should still work. That way the code is easier to test and debug, since the first and last line could be temporarily commented out. There are of course simpler ways to accumulate results, for later output without shown input. But this way should preserve aspects of the flow like warnings and userinfo, etc. acer

It's crude, but... Could you put all the commands in a string, inside a single collapsed Section. Then, also inside the collapsed section, define a procedure to parse and evaluate the pieces of that string as maple commands. Then, outside the section, in the open air, run that procedure against that string. Then there's very little to delete. Eg,

H:="sin(x);|int(cos(x),x);|plot(x^2,x=1..2);|3*x^2-cot(x);|h;"; Q:=proc(s::string) local i; for i in StringTools:-Split(s,"|") do print(eval(parse(i))); od; end proc; Q(H); Then collapse the Section. There's just the Q(H) as visible input. acer

It's a good idea to check the final results of any computation, regardless of whether it comes from a CAS or some other program. My emphasis would be on the word "final", there. A particular definite integral result might well not be the final result. If the software double checks all intermediate results, then it'll be far slower. Why not check only the final result, if it is a compound and more involved computation? If problems are found with the final result, then go ahead and attempt a forced test of intermediate results. There's another important issue with numerical floating point tests of symbolic results. If you don't know the numerical conditioning of the problem, how are you going to know what fixed precision at which to do the computation? How will you decide, for example, that small imaginary floating-point components are not merely artefacts? Letting the program try to figure this out has at least two major snags. The first is that it may be very difficult to compute the conditioning (and hence also the needed working precision to get a desired accuracy). The second is that if the program is allowed to automatically adjust the precision then it might try to use an absurd number of working digits (arbitrary, since the example is unspecified). A check of final results does not necessarily have to be a quantitative test, such as a floating-point numeric comparison. It could be a qualitative test, according to some characteristic that is deduced. All software has bugs. Most all computational software has a history of producing some incorrect results. Even an exact definite integral result, corroborated by a floating-point approximation, should be checked somehow if it's going to be used for some important purpose. Adding an automatic floating-point verificaton attempt, with its resulting performance penalty, should be up to the individual to select. It would be a judgment, weighing the liklihood of error against the performance cost. acer

The problem with numerical verification, done automatically, is that it makes the system much slower. It's pretty straightforward for the end-user to get a floating-point verification out of Maple, for exact definite integration examples. But if int() always did it, then it'd be measurably slower. Many people might be unhappy about that. Leaving it as a choice for the user, after the computation, is nicer. The idea is very good for one related purpose, however. It is very good as for assuring that int()'s exact definite integration schemes are behaving and performing well. Produce a scheme for automatically generating "random" exact definite integration test problems, and such a scheme can becomes strong test engine. By "random" I mean selections from a true miscellany. acer

I wonder, how many of the issues mentioned in Davenport's 2003 Calculemus paper are still relevant today? (It mentions some other methods done by Maple, Alex, such as table-lookup and convolution of MeijerG functions.) Also, one year after an earlier mapleprimes post, how much of its content is still to the point? acer

DJ, I use ArrayTools:-Alias quite often, but I might never have though of what you did in IR4. That is very neat and very, very efficient. Since ArrayTools:-Alias produces a new rtable, without actually copying the data, it costs very little. It's not even clear that it'd be much benefit to "scatter-point index" directly on M (even if one could thus access the antidiagonal all at once, without a loop). It's possible to offload most of the indexing to the time of element access, using an indexing function. Of course, this induces a huge penalty at access time (which I don't measure because it depends on the final usage, which includes printing!). Really, I just show this for fun.

`index/antident`:=proc(idx,M,val)
if nargs>2 then return NULL; end if;
if idx[1]=op([1,2],M)-idx[2]+1 then -1 else 0 end if;
end proc:

IR5 := proc(n,{compact::truefalse:=true},$)
   local M;
   M:=Matrix(n,n,shape='antident',storage=empty);
   if not compact then
      Matrix(M);
   else
      M;
   end if;
end proc:

st,ba,bu:=time(),kernelopts(bytesalloc),kernelopts(bytesused):
IR5(100):
#IR5(100,compact=false): # slow
#IR4(100):
time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu;

Something that puzzled me. If I debug the indexing function, and call IR5(1) with a colon to suppress printing, then why is the indexing function called twice!? acer

DJ, I use ArrayTools:-Alias quite often, but I might never have though of what you did in IR4. That is very neat and very, very efficient. Since ArrayTools:-Alias produces a new rtable, without actually copying the data, it costs very little. It's not even clear that it'd be much benefit to "scatter-point index" directly on M (even if one could thus access the antidiagonal all at once, without a loop). It's possible to offload most of the indexing to the time of element access, using an indexing function. Of course, this induces a huge penalty at access time (which I don't measure because it depends on the final usage, which includes printing!). Really, I just show this for fun.

`index/antident`:=proc(idx,M,val)
if nargs>2 then return NULL; end if;
if idx[1]=op([1,2],M)-idx[2]+1 then -1 else 0 end if;
end proc:

IR5 := proc(n,{compact::truefalse:=true},$)
   local M;
   M:=Matrix(n,n,shape='antident',storage=empty);
   if not compact then
      Matrix(M);
   else
      M;
   end if;
end proc:

st,ba,bu:=time(),kernelopts(bytesalloc),kernelopts(bytesused):
IR5(100):
#IR5(100,compact=false): # slow
#IR4(100):
time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu;

Something that puzzled me. If I debug the indexing function, and call IR5(1) with a colon to suppress printing, then why is the indexing function called twice!? acer

Does it speed it up, to have the inner assignment done by another pair of explicit loops, instead of creating many new scaled copies of Matrix b?

KroneckerProduct2 := proc(a::Matrix,b::Matrix)
        local i,j,aRow,aCol,bRow,bCol,p,k,l;
        aRow,aCol := LinearAlgebra:-Dimension(a);
        bRow,bCol := LinearAlgebra:-Dimension(b);
        p := Matrix(aRow * bRow,aCol * bCol);
        for i from 1 to aRow do
        for j from 1 to aCol do
                for k from 1 to bRow do
                for l from 1 to bCol do
                    p[(i-1)*bRow+k,(j-1)*bCol+l]:=a[i,j]*b[k,l];
                end do;
                end do;
        end do;
        end do;
        p
end proc:

Hopefully I did it correctly. All those a[i,j]*b Matrices are collectible garbage. With enough of them produced, Maple can slow right down. acer

Some person's been doing that, now and then, for months. The admins clear off the accounts pretty quickly. It's always some random name. I suspect that it's done by creating blog entries and then deleting them, in such a way that the points are retained. I haven't figured out how, exactly.. When I reload and notice that it's active, sometimes I try to get a glimpse of a blog entry, so that I can reply and so "fix" it as uneditable. I've never seen an entry though, so I guess that they get removed quite quickly. I've never tried subscribing to the RSS feed, though. I wonder why the admins haven't fixed the loophole. acer

I find the documentation for CodeTools[Profiling] to be obscure. But nprofile, that is easier (and it is easier still, if one also reads the help-page for exprofile). Maybe I rely too much on the debugger. But the profiling did show some interesting things to look for inside the debugger. Here are some things which don't look so right. The procedure returned by, RandomTools:-Generate('distribution(Normal(0, 1))', makeproc = true)) will itself call Statistics:-Sample(distribution). But, as discussed in other threads recently on mapleprimes, that call to Sample() will in turn create and return a new distinct procedure! Surely that is the wrong way to use Sample(), in this circumstance. My initial surmise turns out to be true after all -- this usage is resulting in a new randon number generating procedure being created for each entry in the final Vector. Let's look at the procedure returned by RandomTools:-Generate. Instead of the way it does, answer := Statistics:-Sample(distribution); answer(1)[1] it would be better to do, Statistics:-Sample(distribution,1)[1]; Even that is not perfect, as it creates a new 1-Vector for each call. Why can't RandomTools:-Generate call Statistics:-Sample(distribution) just once, and stuff the resulting procedure into the proc that RandomTools:-Generate returns? I can't see how it isn't seriously wrong for RandomTools:-Generate(...,makeproc=true) to create a procedure which calls Statistics:-Sample(distribution) each time it's invoked. Another unfortunate thing is this. Every time Sample(distribution) is done, it calls ProcessDistribution(). And each time ProcessDistribution() is called, that in turn calls Statistics:-Distributions:-Inventory:-ListKnownDistributions. And ListKnownDistributions() does a lexorder sort on all the distributions in DISTRIBUTION_TAB. 33% of the memory usage in the original submitter's example is undergone inside ListKnownDistributions(). Isn't that a problem in ProcessDistribution() more generally? Couldn't the DISTRIBUTION_TAB be ordered up front. If anything ever added to it, then it could be re-ordered only then. acer

These are just ideas for amending Sample to act inplace. They don't actually work. Joe, the ability to handle ranges nicely, which you recently showed for the copy case, could be trickier. with(Statistics): X := RandomVariable(Normal(1,0)): # This next could operate inplace on a Vector argument, # and fill its first N entries with values. S:=Sample(X,N,V); # This next could return a procedure that expects # two arguments of types posint,Vector. S:=Sample(X,inplace=true); S(N,V); # This next could be the same as the current Sample(X). S:=Sample(X,inplace=false): S(N); An optional offset= parameter is also possible, as opposed to a range. It would look slightly awkward, to allow an offset for dealing with the inplace Vector when that's not currently supported in the copy (current, non-inplace) implementation. Also, if were to take an offset option, then why not a stride option too? Thinking about it a bit harder, maybe neither offset nor stride might be implemented since one might be able to modify the programmatic access to the entries to get the same effects? acer

Very nice, as proof of concept, and in performance. > (st,ba,bu):=time(),kernelopts(bytesalloc),kernelopts(bytesused): > test4(30000,100): > time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu; 0.322, 3276200, 3816512 Even the popular C rng's re-use arrays like this. Your code shows the benefit so directly. Both the speed and memory usage are lower. With normal use of current Statistics, it's only possible to get one of the two so low while the other remains several powers of 2 greater. An interesting question then becomes: should Statistics:-Sample(X) return a proc that allowed an optional Vector argument, or would processing for that option add too much overhead? Would it better to have a new routine like SampleRtable(X) which returned a proc which expected exactly two arguments? What's the tradeoff, between the confusion of more user-level routines and the ability to get very low overhead generated procedures? I mention the idea of two arguments to the procedure returned by SampleRtable(X), since one might only want to generate less random numbers than fit in the rtable. acer

That's quite neat, Joe. Looking at the help-page, ?Statistics,Sample , I don't see where even the simpler invocation Sample(X) is shown. When it is called as Sample(X), for a random variable X, then omitting the second parameter N will result a procedure being returned. That procedure can itself be polled, to return separate Vector samples. This usage appears in several of the examples higher up in this thread. The first person to show it may have been user "alex". The ?Statistics,Sample help-page could be filled with more details on these otherwise hidden bits of functionality. I wonder what other eggs may be found in Statistics. acer

You are quite right, it does create just one generator procedure, sorry. When using the RandomTools generator, more Maple function calls are done to fill in each vector element, and that is also overhead. But garbage collection does indeed happen here, as evinced by the big difference between bytesused and bytesalloc and the regular memory usage "updates". I notice that you omitted the datatype=float[8] in your testing loop. If you add it in, then that might give some assurance that the memory use is not to store many software float Vectors. It doesn't clear it all up, by far, but adding it back might help lay the cause elsewhere. Why bytesalloc keeps growing, through your testing loop, when garbage collections are happening, that I don't know. It might need expert knowledge of inner workings of the garbage collection mechanism to explain it properly. I think that Statistics rather than RandomTools is the better tool for this sort of computation with many generated random numbers. I don't see how to use RandomTools:-Generate to produce a hardware datatype Vector without incurring the overhead of production of many intermediate software float objects or of multiple Maple function calls per element. acer