Investigation of the code called by Statistics:-ScatterPlot reveals that the xerrors and yerrors optional arguments are not taken into account when calling NonlinearFit (ie, when the fit parameter is also specified to ScatterPlot).

This can be confirmed by stepping through computation inside Maple's debugger. The relevant routines may be debugged with these preliminary commands:

kernelopts(opaquemodules=false):
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStruct);                                                                        
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStructTab[default]);                                                                                
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStructTab[':-errors']);                                                                          
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStructTab[':-fit']);

In the case where yerrors is supplied and xerrors is not supplied (error in the dependent variable data only) then weighted least squares might be tried. See here and here. The second of those links may show the way to compute a set of weights which may be passed to Statistics:-NonlinearFit. Doing this by hand (since ScatterPlot doesn't do it) might make a nice blog post. But the definition of what yerrors is supposed to represent could be in question. Right now it is used by ScatterPlot to produce plotted lines through the data points which indicate the possible spread at each plotted data point. But if instead it were taken to indicate the variances of each (presumably uncorrelated) measurement then appropriate least squares weights might be computed. Assuming that the supplied data points in argument Y represent ed the means, and assuming a reasonable distribution, then with this new definition of yerrors the spread could still be displayed (as say a wide, specific confidence interval). It'd get more complicated if the measurements' errors were correlated. Even in the uncorrelated case it would have to be checked that NonlinearFit's handling of its weights parameter conforms to the weight appearing in the normal equations in the reference link. (If that's not true then there not much sense in computing weights from the variance.) I haven't investigated that.

If there is (measurement, say) error in both the dependent and independent variables' data then a total least squares approach may be useful. This naturally brings up the case where xerrors and yerrors are present,

Getting one's hands on the actual parameter values computed for the nonlinear fit as done by ScatterPlot would be an enhancement. (Even a raised infolevel doesn't show it.) Having that enhancement, when xerrors and yerrors are supplied, would be nicer still. And having NonlinearFit itself also accept (co-)variance information for its data -- in lieu of weights -- might be nicer still. These are Statistics routines, after all.

acer

Investigation of the code called by Statistics:-ScatterPlot reveals that the xerrors and yerrors optional arguments are not taken into account when calling NonlinearFit (ie, when the fit parameter is also specified to ScatterPlot).

This can be confirmed by stepping through computation inside Maple's debugger. The relevant routines may be debugged with these preliminary commands:

kernelopts(opaquemodules=false):
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStruct);                                                                        
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStructTab[default]);                                                                                
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStructTab[':-errors']);                                                                          
stopat(Statistics:-Visualization:-ScatterPlot:-BuildPlotStructTab[':-fit']);

In the case where yerrors is supplied and xerrors is not supplied (error in the dependent variable data only) then weighted least squares might be tried. See here and here. The second of those links may show the way to compute a set of weights which may be passed to Statistics:-NonlinearFit. Doing this by hand (since ScatterPlot doesn't do it) might make a nice blog post. But the definition of what yerrors is supposed to represent could be in question. Right now it is used by ScatterPlot to produce plotted lines through the data points which indicate the possible spread at each plotted data point. But if instead it were taken to indicate the variances of each (presumably uncorrelated) measurement then appropriate least squares weights might be computed. Assuming that the supplied data points in argument Y represent ed the means, and assuming a reasonable distribution, then with this new definition of yerrors the spread could still be displayed (as say a wide, specific confidence interval). It'd get more complicated if the measurements' errors were correlated. Even in the uncorrelated case it would have to be checked that NonlinearFit's handling of its weights parameter conforms to the weight appearing in the normal equations in the reference link. (If that's not true then there not much sense in computing weights from the variance.) I haven't investigated that.

If there is (measurement, say) error in both the dependent and independent variables' data then a total least squares approach may be useful. This naturally brings up the case where xerrors and yerrors are present,

Getting one's hands on the actual parameter values computed for the nonlinear fit as done by ScatterPlot would be an enhancement. (Even a raised infolevel doesn't show it.) Having that enhancement, when xerrors and yerrors are supplied, would be nicer still. And having NonlinearFit itself also accept (co-)variance information for its data -- in lieu of weights -- might be nicer still. These are Statistics routines, after all.

acer

You seem to have this idea that asking Maple's fsolve to return a value of t for which the derivative of Y is zero should necessarily result in a global maximum of Y over your interval 4..6.

That is not right.

A smooth differentiable function (here: Maple expression) of one real variable will attain its global maximum either at one of the end points or at a point at which the derivative equals zero. That does not mean that any point at which the derivative is zero is necessarily the global maximum.

All you did initially was ask fsolve for any point in the range 4..6 at which the derivative was zero. (The fsolve routine is a root-finder not a maximizer and so you were not telling maple that  of all the possible zero slope points you wanted the one which maximized the expression.) Four different cases make it not follow that such a point is the global maximum. Take z as being one of the seven points inside 4..6 for which the derivative of Y is zero (ie. evaluated when t=z). Any of these four situations below could hold, for z not a global maximum over the interval.

• z could be a minimum
• z could be only a local maximum
• z could be an inflection point
• eval(Y,t=z) could be less than Y evaluated at either of the end-points 4 or 6.

So hopefully that clears up why using fsolve to solve for diff(Y,t)=0 is not by itself logically enough to specify a global maximum. Maple's response was not wrong. The question that you posed it was wrong for getting the result that you expected.

Now consider the routine Optimization:-Maximize. It's help-page specifically says that it usually only finds a local maximum, and doesn't promise to find the global maximum. And in the single variable case (like that of your Y) an optional argument can enable a different algorithm whose purpose is to try to find the global maximum.

> Y:=a*t^2*(exp(-b*t))*cos(c*t)/(1+d*sqrt(t)):
> infolevel[Optimization]:=1:

> Optimization:-Maximize(Y,t=4..6);
NLPSolve:   calling NLP solver
SolveUnivariate:   using method=quadratic
SolveUnivariate:   number of problem variables
               [2.27131144225468473, [t = 5.65403157341939799]]
 
> Optimization:-Maximize(Y,t=4..6,method=branchandbound);
NLPSolve:   calling NLP solver
SolveUnivariate:   using method=branchandbound
SolveUnivariate:   number of problem variables
               [2.40059887590987087, [t = 4.39825720060096792]]

acer

You seem to have this idea that asking Maple's fsolve to return a value of t for which the derivative of Y is zero should necessarily result in a global maximum of Y over your interval 4..6.

That is not right.

A smooth differentiable function (here: Maple expression) of one real variable will attain its global maximum either at one of the end points or at a point at which the derivative equals zero. That does not mean that any point at which the derivative is zero is necessarily the global maximum.

All you did initially was ask fsolve for any point in the range 4..6 at which the derivative was zero. (The fsolve routine is a root-finder not a maximizer and so you were not telling maple that  of all the possible zero slope points you wanted the one which maximized the expression.) Four different cases make it not follow that such a point is the global maximum. Take z as being one of the seven points inside 4..6 for which the derivative of Y is zero (ie. evaluated when t=z). Any of these four situations below could hold, for z not a global maximum over the interval.

• z could be a minimum
• z could be only a local maximum
• z could be an inflection point
• eval(Y,t=z) could be less than Y evaluated at either of the end-points 4 or 6.

So hopefully that clears up why using fsolve to solve for diff(Y,t)=0 is not by itself logically enough to specify a global maximum. Maple's response was not wrong. The question that you posed it was wrong for getting the result that you expected.

Now consider the routine Optimization:-Maximize. It's help-page specifically says that it usually only finds a local maximum, and doesn't promise to find the global maximum. And in the single variable case (like that of your Y) an optional argument can enable a different algorithm whose purpose is to try to find the global maximum.

> Y:=a*t^2*(exp(-b*t))*cos(c*t)/(1+d*sqrt(t)):
> infolevel[Optimization]:=1:

> Optimization:-Maximize(Y,t=4..6);
NLPSolve:   calling NLP solver
SolveUnivariate:   using method=quadratic
SolveUnivariate:   number of problem variables
               [2.27131144225468473, [t = 5.65403157341939799]]
 
> Optimization:-Maximize(Y,t=4..6,method=branchandbound);
NLPSolve:   calling NLP solver
SolveUnivariate:   using method=branchandbound
SolveUnivariate:   number of problem variables
               [2.40059887590987087, [t = 4.39825720060096792]]

acer

The printed message about nonconvergence seems to make most sense outside and after the loop. That's why I moved it outside the loop, because of that purpose. But a test on abs(x[k]-x[k-1]) is not out of place inside the loop, for another purpose. It would also be quite sensible to put such a check inside the loop so that an early return could be made if convergence occurred while k was still less than N. You may not want it to continue with the full N iterations if the tolerance has already been met.

acer

The printed message about nonconvergence seems to make most sense outside and after the loop. That's why I moved it outside the loop, because of that purpose. But a test on abs(x[k]-x[k-1]) is not out of place inside the loop, for another purpose. It would also be quite sensible to put such a check inside the loop so that an early return could be made if convergence occurred while k was still less than N. You may not want it to continue with the full N iterations if the tolerance has already been met.

acer

You need to make sure that x[k] and x[k-1] have been assigned a numeric value, for each time that they are compared and for the value of k at that moment. Make sure that you stick with either a scheme with indexed x[k],x[k-1],etc or a scheme with x,xnew,xold,etc. You were mixing x and indexed x[k], which wouldn't work.

Also, you indicated that you only wanted to print the message if convergence failed for all k from 1 to N, so put it after the loop and not inside the loop.

NR2:=proc(f::mathfunc,x0::complex,N::posint,eps)
> local x,k:
>   x[0] := x0:
>   for k to N do:
>     x[k] := evalf( x[k-1]-f(x[k-1])/D(f)(x[k-1]) );
>   end do;
>   if abs(x[N]-x[N-1]) >= eps then
>     printf("Convergence has not been achieved after %a iterations!\n",N);
>   else
>     return x[N];
>   end if;
> end proc:
>
> f:= x-> x^5-1:
>
> NR2(f,0.6+I*0.6,10,0.00001);
Convergence has not been achieved after 10 iterations!
> NR2(f,0.2+I*0.6,10,0.00001);
                         0.3090169944 + 0.9510565163 I

Side tip: maple's for-loop counters finish with a value incremented one step more than the last used value, when they have finished. For example, a for-loop counting k from 1 to 10 will have value 11 after it's finished. This matters, if you plan to refer to x[k] after it's finished. Notice that I referred to x[N] after the loop. I could also have referred to x[k-1]=x[10] but not to x[k]=x[11] which is unassigned.

Lastly, Robert's suggestion to use evalf was so that a large (potentially huge) symbolic expression did not accumulate via the iterative process. Using evalf can cure that, but only if it's done prior to assigning to x or x[k]. You had it done only as a separate task afterwards. I put it right in the iterative step.

acer

You need to make sure that x[k] and x[k-1] have been assigned a numeric value, for each time that they are compared and for the value of k at that moment. Make sure that you stick with either a scheme with indexed x[k],x[k-1],etc or a scheme with x,xnew,xold,etc. You were mixing x and indexed x[k], which wouldn't work.

Also, you indicated that you only wanted to print the message if convergence failed for all k from 1 to N, so put it after the loop and not inside the loop.

NR2:=proc(f::mathfunc,x0::complex,N::posint,eps)
> local x,k:
>   x[0] := x0:
>   for k to N do:
>     x[k] := evalf( x[k-1]-f(x[k-1])/D(f)(x[k-1]) );
>   end do;
>   if abs(x[N]-x[N-1]) >= eps then
>     printf("Convergence has not been achieved after %a iterations!\n",N);
>   else
>     return x[N];
>   end if;
> end proc:
>
> f:= x-> x^5-1:
>
> NR2(f,0.6+I*0.6,10,0.00001);
Convergence has not been achieved after 10 iterations!
> NR2(f,0.2+I*0.6,10,0.00001);
                         0.3090169944 + 0.9510565163 I

Side tip: maple's for-loop counters finish with a value incremented one step more than the last used value, when they have finished. For example, a for-loop counting k from 1 to 10 will have value 11 after it's finished. This matters, if you plan to refer to x[k] after it's finished. Notice that I referred to x[N] after the loop. I could also have referred to x[k-1]=x[10] but not to x[k]=x[11] which is unassigned.

Lastly, Robert's suggestion to use evalf was so that a large (potentially huge) symbolic expression did not accumulate via the iterative process. Using evalf can cure that, but only if it's done prior to assigning to x or x[k]. You had it done only as a separate task afterwards. I put it right in the iterative step.

acer

You could throw in this below as an option to DEplot. I used the layout palette to obtain the typesetting incantation for x-dot.

labels=[t,typeset(`#mover(mi("x"),mrow(mo("⁢"),mo(".")))`)]

acer

You could throw in this below as an option to DEplot. I used the layout palette to obtain the typesetting incantation for x-dot.

labels=[t,typeset(`#mover(mi("x"),mrow(mo("⁢"),mo(".")))`)]

acer

I find problems like this can be tough to do with Maple.

Ferr:=-10*(.7845815999*u2+3.141592654)*sinh(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*cos(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*u2/((100+fout)^(1/2)*(.1998118316*u2+1))+200*(.7845815999*u2+3.141592654)^2*sinh(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*sin(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))/((100+fout)*(.1998118316*u2+1)^2)+10*(.7845815999*u2+3.141592654)*cosh(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*u2*sin(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))/((100+fout)^(1/2)*(.1998118316*u2+1)):
plots:-implicitplot(Ferr,u2=0..50,fout=-1..1,numpoints=30000, gridlines=true);

These look right, judging from the graph, for the maximum and minimum points,

> Optimization:-Maximize(fout,{Ferr=0},
>        initialpoint=[u2=0,fout=0],u2=5..50,fout=-1..1);

[0.00826786487008719304,
    [u2 = 13.6048493803282895, fout = 0.00826786487008719304]]

> Optimization:-Minimize(fout,{Ferr=0},
>        initialpoint=[u2=0,fout=0],u2=0..10,fout=-1..1);

[-0.0594716927922686461,
    [u2 = 1.19113556326925552, fout = -0.0594716927922686461]]

Maple seemed to need a (feasible?) initial point in order to proceed above.

acer

I find problems like this can be tough to do with Maple.

Ferr:=-10*(.7845815999*u2+3.141592654)*sinh(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*cos(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*u2/((100+fout)^(1/2)*(.1998118316*u2+1))+200*(.7845815999*u2+3.141592654)^2*sinh(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*sin(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))/((100+fout)*(.1998118316*u2+1)^2)+10*(.7845815999*u2+3.141592654)*cosh(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))*u2*sin(10*(.7845815999*u2+3.141592654)/((100+fout)^(1/2)*(.1998118316*u2+1)))/((100+fout)^(1/2)*(.1998118316*u2+1)):
plots:-implicitplot(Ferr,u2=0..50,fout=-1..1,numpoints=30000, gridlines=true);

These look right, judging from the graph, for the maximum and minimum points,

> Optimization:-Maximize(fout,{Ferr=0},
>        initialpoint=[u2=0,fout=0],u2=5..50,fout=-1..1);

[0.00826786487008719304,
    [u2 = 13.6048493803282895, fout = 0.00826786487008719304]]

> Optimization:-Minimize(fout,{Ferr=0},
>        initialpoint=[u2=0,fout=0],u2=0..10,fout=-1..1);

[-0.0594716927922686461,
    [u2 = 1.19113556326925552, fout = -0.0594716927922686461]]

Maple seemed to need a (feasible?) initial point in order to proceed above.

acer

The term least squares is used to refer to a method for solving various different problems. Roughly, it means minimizing a sum of squares (usually of differences).

In this case, you indicated that you wanted to use it as a method for finding a line of best fit. The two choices of routine that I showed can both serve this purpose of fitting a line to data. The results they returned are both the equations of a line, ie. p*t+q , which is the form you requested. (I couldn't make it p*x+q because you had already assigned to the name x.)

But there is also, for example, least squares as a means of solving an overdetermined system of linear equations. Indeed, this can be the way that the abovementioned fitting computation can be done, behind the scenes. If you really wanted to, you could figure out how to use your data to construct such an overdetermined linear system, and then call Optimization:-LSSolve on it, and then re-interpret the Vector result to get the equation of the line. I guessed that you'd prefer having one of those two fitting routines do all that bookkeeping for you.

acer

The term least squares is used to refer to a method for solving various different problems. Roughly, it means minimizing a sum of squares (usually of differences).

In this case, you indicated that you wanted to use it as a method for finding a line of best fit. The two choices of routine that I showed can both serve this purpose of fitting a line to data. The results they returned are both the equations of a line, ie. p*t+q , which is the form you requested. (I couldn't make it p*x+q because you had already assigned to the name x.)

But there is also, for example, least squares as a means of solving an overdetermined system of linear equations. Indeed, this can be the way that the abovementioned fitting computation can be done, behind the scenes. If you really wanted to, you could figure out how to use your data to construct such an overdetermined linear system, and then call Optimization:-LSSolve on it, and then re-interpret the Vector result to get the equation of the line. I guessed that you'd prefer having one of those two fitting routines do all that bookkeeping for you.

acer

As already mentioned above, the lowess option of ScatterPlot does a form of weighted least squares. And a Vector of weights may be provided to NonlinearFit. It may be useful to think about the differences of these two approaches. An interesting issue is the possible availability of the fitted function and all its computed parameter values.

The way to supply weights to NonlinearFit is clear from its help-page which describes the weights option for this. I don't quite inderstand how those weights are then used, as weights don't seem to be an option for Optimization:-LSSolve. I understand that in weighted least squares problems with data errors it is usual for such weights to be taken using variance of the data. But I don't know exactly how the Maple solver works here. What I suspect is that xerrors and yerrors optional parameters of ScatterPlot may be used to compute weights to be passed on to NonlinearFit. I haven't confirmed this.

It's not clear from the ScatterPlot help-page exactly how the weights for lowess smoothing are chosen. Its three options related to the lowess smoothing are degree, robust, and lowess. It's not clear from that help-page in what way (if any) the xerrors or yerrors options may tie into weighting. I suspect that the don't relate at all. And then there is the question of whether a formulaic fitting result is wanted, since the lowess method will not make that available. The lowess method uses a series of weighted least squares for different points, where weights are used to modify the influence of near neighboring points (rather than to correct for measurement uncertainty directly). I now believe that this is not what the original poster wants.

So here's a question. When passing xerrors and yerrors data to ScatterPlot, when supplied with the fit option, is estimated variance of that extra data used to produce the weights which are then passed along to NonlinearFit? Tracing the Maple computation in the debugger might show whether this is true. If it is, then it may be possible to extract the method for doing it "by hand". In such a way, it may be possible to extract the parameter values that result from the nonlinear fit.

I know that, when calling ScatterPlot with the fit option, Statistics:-NonlinearFit is called, and that Optimization:-LSSolve is also called. It remains to figure out exactly how xerrors and yerrors are used, and whether they modify the above to produce weights for NonlinearFit.

acer