I do not know the reason why  applyrule  does not work, because the documentation for this command is very poor. Use  evalindets  command instead:

restart;
eq1 := a[n+2]+4*a[n+1]+10*a[n]-5*n-1;
evalindets(eq1, `indexed`, t->op(t)*p+q+b[op(t)]);

Try

D[1](u)(0,t)=0, D[1](u)(L,t)=0

Addition.  I think that your teacher's proposal is incorrect in any version of Maple. Here is an example:

restart;
u:=(x,t)->x^2+3*x*t+t^4:
diff(u(0,t),x), diff(u(L,t),x);  # An incorrect way
# Two valid options:
D[1](u)(0,t), D[1](u)(L,t);   # The first way (the best one)
eval(diff(u(x,t),x), x=0), eval(diff(u(x,t),x), x=L);   # The second way

The notation that you are proposing is also incorrect in Maple, because derivative must first be calculated at an arbitrary point. Here is an valid way:

restart;
Du:=diff(u(x,t),x);
eval(Du, x=0);

Formally to extract the sequence of used arithmetic operators from  an expression you can as follows:

a:=3*x^3-5*x^2+3*y:
S:=convert(a,string):
Operators:={"+", "-", "*", "^"}: 
StringTools:-Select(s->is(s in Operators), S);
seq(s, s=%);
ListTools:-Collect([%]);

Edit.

Give these points names  A, B, C, E  that mean their coordinates:

curve := y^2 = x^3 - 43*x + 166;
Points:=[[3,8],[-5,16],[11,32],[3,-8]]:
A, B, C, E:= op(Points);
plots:-display([
plot(+sqrt(rhs(curve)),x = -10..12),
plot(-sqrt(rhs(curve)),x = -10..12),
plots:-pointplot([A, B, C, E],symbol = solidbox)
]);

To label these points on the plot, use plots:-textplot command.

Addition. After you give names to these points, when using plots:-textplot command, you do not need to retype their coordinates (you can just refer to their names):

restart;
curve := y^2 = x^3 - 43*x + 166:
Points:=[[3,8],[-5,16],[11,32],[3,-8]]:
Labels:=["A","B","C","E"]:
plots:-display([
   plot(+sqrt(rhs(curve)),x = -10..12),
   plot(-sqrt(rhs(curve)),x = -10..12),
   plots:-pointplot(Points, symbol = solidbox, color=[green,red,    blue,yellow], symbolsize=17), 
   plots:-textplot([seq([op(Points[i]),Labels[i]], i=1..4)],  align=above, font=[roman,16])
]);
 

Edit.

If you want Maple not to open the parentheses, then do this:

z:=-x-y*I;
(-1)*``(-z);
                            z := -x - y I
                             - (x + y I)

or immediately type

z:=-``(x+y*I);
                           z := - (x + y I)

If later you want Maple to open parentheses, then use  expand command:

z:=-``(x+y*I);
expand(z);
                                 -x - y I

This way you can always сarry the desired factor out of brackets of some  expression, for example:

P:=-3/2*x^2-x+5/2:
-1/2*``(P/(-1/2));
                                        -1/2*(3*x^2+2*x-5)

Edit.

The domain of definition of your function is a semicircle. Use the direct plotting in this domain:

plot3d(4*x*y^2-x^2, x=0..1, y=-sqrt(1-x^2)..sqrt(1-x^2), scaling=constrained, numpoints=10000, axes=normal);

Another option is the using of the polar coordinates:

plot3d(eval([x,y,4*x*y^2-x^2],[x=r*cos(t),y=r*sin(t)]), r=0..1, t=-Pi/2..Pi/2, scaling=constrained, axes=normal);

VerticesOfCuboid:=proc(V1::list, V2::list)
[V1, seq(subsop(i=V2[i], V1), i=1..3), V2, seq(subsop(j=V1[j], V2), j=1..3)];
end proc: 

Example of use:

VerticesOfCuboid([1,1,2], [3,4,5]);

               [[1, 1, 2], [3, 1, 2], [1, 4, 2], [1, 1, 5], [3, 4, 5], [1, 4, 5], [3, 1, 5], [3, 4, 2]]

Addition. In geom3d you can specify a cuboid by  parallelepiped  command and find its parameters by other commands (detail, vertices, faces  and so on).

There are several ways to do this. Here are two ones:

restart;
f := unapply(x*(8*x^2+5*x+cos(y)), x, y);
p := solve({diff(f(x, y), x) = 0, diff(f(x, y), y) = 0}, {x, y});
eval([x,y], p[1]);  # The first way
assign(p[1]);  # The second way 
x, y; 

I understood  this  to mean that  P  is an operator in the functional space and therefore we can do so:

P:=u->(i,j)->u(i+1,j);

Example of use:

u:=(i,j)->i^2+j^2;
v:=P(u);  # v is a new function
v(2,3);
                                    
 

For  Q  everything is similar.

Example:

y:=t-><sin(t), cos(t), exp(2*t)>;
series~(y(t), t, 3);

For the example, I have specified values for  u, L, c  :

restart;
z:=(-1/(4*(-u+L)))*(4*p^2*b^2*d-3*g*c*p^2);
u:=2:  L:=1:  c:=0.6: 
constr:={ 1-(2*d)/(p*(sqrt(g)+1))<=b, b<=1, 0<=d, d<=L, c<=p, p<=u, 1<=g, g<=4};  
Optimization:-Maximize(z, constr);

Edit.

You can use  `A=B=C`  instead. Output will be without any quotes.

If you need to use this as a logical condition (for example inside a selection statement  if ... fi), then write  A=B and B=C

Try (only for purely visual purposes)

exp(`2 t`) ;

This is a workaround for Classic Interface. In Standard one there are no any parentheses around the exponent.

Edit.

Use  InertForm  package.

Example:

A:=<1,2; 3,4>:
B:=<3,4; 1,2>:
with(InertForm):
L := Parse("A*B"):
Display(L) = A.B;

A+`&lambda;I`=<1,2; 3,4>;  # Or

A+`&lambda; I`=<1,2; 3,4>;