That does sound more likely. I'm not so good a guesser. I wonder whether in future it might be possible to get jump discontinuties to be shown by dashed lines, like appear in so many texts. Is there already an easy way to get that, does anyone know? It might look nice if jumps from a curve to a finite point, or vertical asymptotes, could get shown as dashed lines though some nice option such as 'discont'='dashed'. acer

We might only guess you think is wrong with it. Is it because there are no vertical bars indicating the jumps at -Pi and Pi? If so, you might try extending the range a tiny amount on each side, so as to help Maple realize that there are jumps there. plot( cot(x), x=-Pi-0.001..Pi+0.001, view=[-Pi..Pi,-3..3] ); acer

It might be useful to know how bad the conditioning can be. Does it get worse, as the size of problems in your class grows? (For example, the condition number for solving linear systems with the so-called Hilbert Matrix will grow with the size N.) I ask because, if you needed 128 decimal digits at smaller sizes, and if the conditioning gets worse as the size grows, then for size 7000 the conditioning might be so much worse. If you can generate problems from your "class" in sizes of multiples of 10, say, then you could set Digits high and look for a pattern. Something like this,

Digits:=500:
kernelopts[printbytes=false]:
with(LinearAlgebra):
for k from 1 to 20 do
M := # however you construct the example,
# with size k*10 by k*10
ConditionNumber(M):print(evalf[10](%));
od:

If above shows that the condition number grows with the size, then you might need a very high working precision to deal with the 7000x7000 case. The required working precision might be prohibitively high. There are iterative sparse solvers for high precision floating-point linear systems, available in LinearSolve. The method='SparseIterative' option forces use of (only) such methods. Specifying the method means that it won't fall back to any other (much slower) method if it encounters difficulties. You might experiment with using that on a small problem in the same class. You could try a smaller system at both hardware and software precision, to compare the performance effect of switching to software precision. Be prepared to see a 15-20 times slowdown just be switching between Digits=14 and Digits=16, which covers the hardware double precision cutoff. Further increase to Digits will result in a further (gradual but steady) slowdown. See the help-page ?IterativeSolver for more details on using this method. That page describes symmetric problems only, but experimentation reveals that there is also a nonsymmetric iterative solver. If you go that route, be sure to create your Matrix directly with datatype=sfloat. You may need to use ImportMatrix to effectively get the data into an sfloat Matrix in the first place (it depends on how sparse it is). Do not(!) try to do general computations with such a Matrix, Maple might try to copy it to a dense Matrix and get bogged down. That includes calling ConditionNumber(). I'd suggest setting infolevel[LinearAlgebra]:=2 and breaking any computation that didn't show a NAG f11 function in the printed progress output. Any attempt to form the dense "rectangular" storage version of your large sparse Matrix might end up exceeding your memory resources. The few things that you might reasonably do with such a huge sparse high-precision floating-point system include,

• linear solving
• matrix-vector multiplication

You might also need to know the answers to questions like this. What sort of accuracy are you after? What would characterize a valid solution for you? (forward error? backward error?) How long are you prepared to wait for result? Do you need to solve for multiple right-hand-sides (Matrix b instead of Vector b, in A.x=b)? acer

Hi Georgios, I can't seem to reproduce the problem with Browse, on my 11.02 in 64bit Linux, sorry. But I can answer your second question. The setting interface(rtablesize) controls the cutoff size below which one only sees that "summary" detail of the Vector/Matrix/Array. acer

I just realized that the commands in the string could all be written out more nicely, in one execution group, as say, H := " sin(x); int(cos(x),x); plot(x^2,x=1..2); x^3; ": If the string is then split at the symbol "\n", then it should still work. That way the code is easier to test and debug, since the first and last line could be temporarily commented out. There are of course simpler ways to accumulate results, for later output without shown input. But this way should preserve aspects of the flow like warnings and userinfo, etc. acer

It's crude, but... Could you put all the commands in a string, inside a single collapsed Section. Then, also inside the collapsed section, define a procedure to parse and evaluate the pieces of that string as maple commands. Then, outside the section, in the open air, run that procedure against that string. Then there's very little to delete. Eg,

H:="sin(x);|int(cos(x),x);|plot(x^2,x=1..2);|3*x^2-cot(x);|h;"; Q:=proc(s::string) local i; for i in StringTools:-Split(s,"|") do print(eval(parse(i))); od; end proc; Q(H); Then collapse the Section. There's just the Q(H) as visible input. acer

It's a good idea to check the final results of any computation, regardless of whether it comes from a CAS or some other program. My emphasis would be on the word "final", there. A particular definite integral result might well not be the final result. If the software double checks all intermediate results, then it'll be far slower. Why not check only the final result, if it is a compound and more involved computation? If problems are found with the final result, then go ahead and attempt a forced test of intermediate results. There's another important issue with numerical floating point tests of symbolic results. If you don't know the numerical conditioning of the problem, how are you going to know what fixed precision at which to do the computation? How will you decide, for example, that small imaginary floating-point components are not merely artefacts? Letting the program try to figure this out has at least two major snags. The first is that it may be very difficult to compute the conditioning (and hence also the needed working precision to get a desired accuracy). The second is that if the program is allowed to automatically adjust the precision then it might try to use an absurd number of working digits (arbitrary, since the example is unspecified). A check of final results does not necessarily have to be a quantitative test, such as a floating-point numeric comparison. It could be a qualitative test, according to some characteristic that is deduced. All software has bugs. Most all computational software has a history of producing some incorrect results. Even an exact definite integral result, corroborated by a floating-point approximation, should be checked somehow if it's going to be used for some important purpose. Adding an automatic floating-point verificaton attempt, with its resulting performance penalty, should be up to the individual to select. It would be a judgment, weighing the liklihood of error against the performance cost. acer

The problem with numerical verification, done automatically, is that it makes the system much slower. It's pretty straightforward for the end-user to get a floating-point verification out of Maple, for exact definite integration examples. But if int() always did it, then it'd be measurably slower. Many people might be unhappy about that. Leaving it as a choice for the user, after the computation, is nicer. The idea is very good for one related purpose, however. It is very good as for assuring that int()'s exact definite integration schemes are behaving and performing well. Produce a scheme for automatically generating "random" exact definite integration test problems, and such a scheme can becomes strong test engine. By "random" I mean selections from a true miscellany. acer

I wonder, how many of the issues mentioned in Davenport's 2003 Calculemus paper are still relevant today? (It mentions some other methods done by Maple, Alex, such as table-lookup and convolution of MeijerG functions.) Also, one year after an earlier mapleprimes post, how much of its content is still to the point? acer

DJ, I use ArrayTools:-Alias quite often, but I might never have though of what you did in IR4. That is very neat and very, very efficient. Since ArrayTools:-Alias produces a new rtable, without actually copying the data, it costs very little. It's not even clear that it'd be much benefit to "scatter-point index" directly on M (even if one could thus access the antidiagonal all at once, without a loop). It's possible to offload most of the indexing to the time of element access, using an indexing function. Of course, this induces a huge penalty at access time (which I don't measure because it depends on the final usage, which includes printing!). Really, I just show this for fun.

`index/antident`:=proc(idx,M,val)
if nargs>2 then return NULL; end if;
if idx[1]=op([1,2],M)-idx[2]+1 then -1 else 0 end if;
end proc:

IR5 := proc(n,{compact::truefalse:=true},$)
   local M;
   M:=Matrix(n,n,shape='antident',storage=empty);
   if not compact then
      Matrix(M);
   else
      M;
   end if;
end proc:

st,ba,bu:=time(),kernelopts(bytesalloc),kernelopts(bytesused):
IR5(100):
#IR5(100,compact=false): # slow
#IR4(100):
time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu;

Something that puzzled me. If I debug the indexing function, and call IR5(1) with a colon to suppress printing, then why is the indexing function called twice!? acer

DJ, I use ArrayTools:-Alias quite often, but I might never have though of what you did in IR4. That is very neat and very, very efficient. Since ArrayTools:-Alias produces a new rtable, without actually copying the data, it costs very little. It's not even clear that it'd be much benefit to "scatter-point index" directly on M (even if one could thus access the antidiagonal all at once, without a loop). It's possible to offload most of the indexing to the time of element access, using an indexing function. Of course, this induces a huge penalty at access time (which I don't measure because it depends on the final usage, which includes printing!). Really, I just show this for fun.

`index/antident`:=proc(idx,M,val)
if nargs>2 then return NULL; end if;
if idx[1]=op([1,2],M)-idx[2]+1 then -1 else 0 end if;
end proc:

IR5 := proc(n,{compact::truefalse:=true},$)
   local M;
   M:=Matrix(n,n,shape='antident',storage=empty);
   if not compact then
      Matrix(M);
   else
      M;
   end if;
end proc:

st,ba,bu:=time(),kernelopts(bytesalloc),kernelopts(bytesused):
IR5(100):
#IR5(100,compact=false): # slow
#IR4(100):
time()-st,kernelopts(bytesalloc)-ba,kernelopts(bytesused)-bu;

Something that puzzled me. If I debug the indexing function, and call IR5(1) with a colon to suppress printing, then why is the indexing function called twice!? acer

Does it speed it up, to have the inner assignment done by another pair of explicit loops, instead of creating many new scaled copies of Matrix b?

KroneckerProduct2 := proc(a::Matrix,b::Matrix)
        local i,j,aRow,aCol,bRow,bCol,p,k,l;
        aRow,aCol := LinearAlgebra:-Dimension(a);
        bRow,bCol := LinearAlgebra:-Dimension(b);
        p := Matrix(aRow * bRow,aCol * bCol);
        for i from 1 to aRow do
        for j from 1 to aCol do
                for k from 1 to bRow do
                for l from 1 to bCol do
                    p[(i-1)*bRow+k,(j-1)*bCol+l]:=a[i,j]*b[k,l];
                end do;
                end do;
        end do;
        end do;
        p
end proc:

Hopefully I did it correctly. All those a[i,j]*b Matrices are collectible garbage. With enough of them produced, Maple can slow right down. acer

Some person's been doing that, now and then, for months. The admins clear off the accounts pretty quickly. It's always some random name. I suspect that it's done by creating blog entries and then deleting them, in such a way that the points are retained. I haven't figured out how, exactly.. When I reload and notice that it's active, sometimes I try to get a glimpse of a blog entry, so that I can reply and so "fix" it as uneditable. I've never seen an entry though, so I guess that they get removed quite quickly. I've never tried subscribing to the RSS feed, though. I wonder why the admins haven't fixed the loophole. acer

I find the documentation for CodeTools[Profiling] to be obscure. But nprofile, that is easier (and it is easier still, if one also reads the help-page for exprofile). Maybe I rely too much on the debugger. But the profiling did show some interesting things to look for inside the debugger. Here are some things which don't look so right. The procedure returned by, RandomTools:-Generate('distribution(Normal(0, 1))', makeproc = true)) will itself call Statistics:-Sample(distribution). But, as discussed in other threads recently on mapleprimes, that call to Sample() will in turn create and return a new distinct procedure! Surely that is the wrong way to use Sample(), in this circumstance. My initial surmise turns out to be true after all -- this usage is resulting in a new randon number generating procedure being created for each entry in the final Vector. Let's look at the procedure returned by RandomTools:-Generate. Instead of the way it does, answer := Statistics:-Sample(distribution); answer(1)[1] it would be better to do, Statistics:-Sample(distribution,1)[1]; Even that is not perfect, as it creates a new 1-Vector for each call. Why can't RandomTools:-Generate call Statistics:-Sample(distribution) just once, and stuff the resulting procedure into the proc that RandomTools:-Generate returns? I can't see how it isn't seriously wrong for RandomTools:-Generate(...,makeproc=true) to create a procedure which calls Statistics:-Sample(distribution) each time it's invoked. Another unfortunate thing is this. Every time Sample(distribution) is done, it calls ProcessDistribution(). And each time ProcessDistribution() is called, that in turn calls Statistics:-Distributions:-Inventory:-ListKnownDistributions. And ListKnownDistributions() does a lexorder sort on all the distributions in DISTRIBUTION_TAB. 33% of the memory usage in the original submitter's example is undergone inside ListKnownDistributions(). Isn't that a problem in ProcessDistribution() more generally? Couldn't the DISTRIBUTION_TAB be ordered up front. If anything ever added to it, then it could be re-ordered only then. acer

These are just ideas for amending Sample to act inplace. They don't actually work. Joe, the ability to handle ranges nicely, which you recently showed for the copy case, could be trickier. with(Statistics): X := RandomVariable(Normal(1,0)): # This next could operate inplace on a Vector argument, # and fill its first N entries with values. S:=Sample(X,N,V); # This next could return a procedure that expects # two arguments of types posint,Vector. S:=Sample(X,inplace=true); S(N,V); # This next could be the same as the current Sample(X). S:=Sample(X,inplace=false): S(N); An optional offset= parameter is also possible, as opposed to a range. It would look slightly awkward, to allow an offset for dealing with the inplace Vector when that's not currently supported in the copy (current, non-inplace) implementation. Also, if were to take an offset option, then why not a stride option too? Thinking about it a bit harder, maybe neither offset nor stride might be implemented since one might be able to modify the programmatic access to the entries to get the same effects? acer