As the error message suggests, see the help page ?alloc.

Usually Maple will use as much of your system's memory as needed, up to the maximum the operating system will allow.  Unless you yourself have limited the memory available to Maple (e.g. with the -T option on the command line), the message probably indicates you're running into a hard limit: there is physically no more memory available.  You say you ran the program successfully before, but then got this error when running it again.  Was this in a new Maple session, or in the same Maple session?  If it's the same Maple session, it's likely that some of the memory used in running the program the first time was still tied up when you tried to run it again, so less memory was available.  Even a restart may not put the memory completely back to its original state: maybe it will for the memory used by the kernel, but not for the GUI. 

On the other hand,  exiting from Maple and starting a new session should free up everything - in that case the blame might attach to other applications or processes that are using up memory.  You might look at your Windows Task Manager (or equivalent on other operating systems) to see what is using memory.  You might be able to shut down some of those to make more memory available.

simplify can do a lot of things, and is generally quite expensive in terms of resources if you allow it to try all the different kinds of simplification available.   It's more efficient to tell simplify to do a particular kind of simplification.  In this case it sounds like you want to use simplify(..., exp).  For example (I didn't try 400000 terms, but hopefully it scales):

> expr:= add(i/(i+1)/exp(i),i=1..40000):
   ti:= time(): Q:= simplify(expr,exp): time()-ti;

37.580

Unfortunately you've hit on two things that Maple doesn't really do yet: "symbolic linear algebra" and proofs.  Maple could verify these identities for n x n matrices for a particular (not too huge) n, but not when n is unspecified.  Thus for the 3 x 3 case:

> with(LinearAlgebra):
   Omega:= Matrix(3,3, symbol=omega);
   U:= Matrix(3,3, symbol=u);
   unitarity:= convert(U . U^%H - IdentityMatrix(3), set);
   simplify(Trace(U . Omega . U^%H) - Trace(Omega), conditions);

0

> A:= Matrix(3,3, symbol=a);
   B:= Matrix(3,3, symbol=b);
   simplify((A.B)^(-1) - B^(-1) . A^(-1));

This requires some analysis first to see what inequalities on a,b,c correspond to the geometrical conditions.  For simplicity let's suppose a >= b >= c >= 0 (the other 5 possible orders of a,b,c are similar).  Then a,b,c form a triangle if they satisfy the triangle inequality: a <= b + c  By the law of cosines, an acute triangle would mean a^2 < b^2 + c^2.  

> P[triangle]:= 6*int(int(int(1, a = b .. min(1,b+c)), b = c .. 1), c = 0 .. 1);

Maple needs a bit of help with this.  It would have worked if we used piecewise instead of min:

> eval(P[triangle],min(1,b+c)=piecewise(b<1-c,b+c,1));

> P[acute]:= 6*int(int(int(1, a = b .. piecewise(b < sqrt(1-c^2),sqrt(b^2+c^2),1)), b = c .. 1), c = 0 .. 1);

I don't know if Maple can do this complicated-looking integral symbolically.  Here's the numerical value

> evalf(%);

0.214601837

> identify(%);

You can (sometimes) get integer solutions of an equation or system using isolve:

> isolve(4*x^3-12*x^2+11*x-3=0);

{x = 1}

But like Axel, I'm puzzled about what you're really trying to do in your example.  If you want to round the solution 5/3 down to the integer 1, you can use floor:

> floor(solve(3*k - 5 = 0));

1

If beta is a rational number, so all the trig functions can be expressed as rational functions of one complex exponential, it should be possible to solve. 
For general beta, I agree with ThU: there won't be a closed-form expression for the answers.
Given numerical values of the parameters, you can use fsolve or RootFinding[Analytic] to find numerical solutions.

As Joe said, more details would help. 

For a system involving polynomials or rational functions in one variable, solve should work.
For example:

> solve({1/x + x <= 3, x + x^2 <= 5});

{RootOf(-5+_Z+_Z^2,index = 2) <= x, x < 0}, {x <= RootOf(-5+_Z+_Z^2,index = 1), RootOf(-3*_Z+1+_Z^2,index = 1) <= x}

> evalf(%);

{-2.791287847 <= x, x < 0.}, {.3819660112 <= x, x <= 1.791287847}

Systems involving non-rational functions are more difficult, just as with equations. 

If you have more than one variable, it's not even clear how one might reasonably represent the solutions.

If you give pointplot a Vector or list of numbers, it will interpret these, in pairs, as coordinates for two-dimensional points.  Thus

pointplot([1,2,3,4]);

would plot the two points [1,2] and [3,4].  For this interpretation to work, the Vector or list must have an even number of elements.

If you wanted to interpret the Vector or list as y coordinates where the x coordinates are 1,2,3,...,n, try listplot.  Thus

listplot([10,20,30], style=point);

will plot the points [1,10], [2,20], [3,30].

Actually you can use Student[Calculus1][ApproximateInt] (which is the replacement for the deprecated student[simpson]) if you define the function properly.  For example:

> n := nops(points);   # note that this must be odd for Simpson's Rule
  f := proc(x) if x::integer then points[x,1] else 'procname'(x) end if end proc;
  Student[Calculus1][ApproximateInt](f(x), x=1..n, partition = (n-1)/2);

But if you really want to do it yourself:

  > 1/3*(points[1,1] + points[n,1]) + 4/3*add(points[2*i,1],i=1..(n-1)/2)
      + 2/3*add(points[2*i-1,1],i=2 .. (n-1)/2);

There are several things wrong with your

G:= x-> x - (F)/(Fd);

 Your F is a function, so you'd need F(x) rather than (F). 

  On the other hand, Fd is an expression, and while it contains x that is not the x that is the formal parameter of G.  You can use unapply to make an expression into a function.  Thus:

G:= unapply(x - F(x)/Fd, x);

Now G is a function, and you use it like any other function, by putting its argument(s) in parentheses after the G.  Thus you could say

x[1]:= evalf(G(x[0]));

and similarly in your loop

x[i]:= evalf(G(x[i-1]));

Finally,

print(accuracy not reached)

won't work because you need quotes " " around a string.  It could be

print("accuracy not reached")

x * f(a,b,c)  evaluates x and f(a,b,c) and multiplies the results.  If you want that to return f(x*a, x*b, x*c) you'll have to define a new version of `*`, not f: f itself should presumably return unevaluated.  This could be done using overloading.  For example:

> M:= module() option package; export `*`;
    `*`:= overload( [ proc(a::algebraic, b::specfunc(anything,f)) option overload;    
                      map2(:-`*`,a,b) end proc ]) 
   end module;
   with(M):

   u * f(2,b);

f(2*u,u*b)

I think the best way to do this is using cylindrical coordinates.

> plot3d([r,theta,4*(r*cos(theta))^2 - (r*sin(theta))^2], r = 0 .. 1, theta=0..2*Pi, 
coords=cylindrical, axes=box, labels=[x,y,z]); 

Try something like this.

with(group);
S3:= permgroup(3,{[[1,2]],[[2,3]]});
L:= elements(S3);
Matrix(6,6,(i,j) -> mulperms(L[i],L[j]));

Another work-around is this:

> thaw(solve( alpha * x - freeze(conjugate(alpha))*x = 1, x));

As Joe suspected, you have typed sL and sCR.  2D Math input does have "implicit multiplication" but you need to have a space between the items you want to multiply, e.g. s L or s C R, otherwise Maple interprets these as multi-letter names rather than products.