Question: problems with ChiSquareSuitableModelTest

Hi, 

The procedure Statistics:-ChiSquareSuitableModelTest returns wrong or stupid results in some situations.
The stupid answer can easily be avoided if the user is careful enough.
The wrong answer is more serious: the standard deviation (in the second case below) is not correctly estimated.

PS: the expression "CORRECT ANSWER" is a short for "POTENTIALLY CORRECT ANSWER" given that what ChiSquareSuitableModelTest really does is not documented
 

restart:

with(Statistics):

randomize():

N := 100:
S := Sample(Normal(0, 1), N):

infolevel[Statistics] := 1:

# 0 parameter to fit from the sample S  CORRECT ANSWER

ChiSquareSuitableModelTest(S, Normal(0, 1), level = 0.5e-1):
print():

Chi-Square Test for Suitable Probability Model
----------------------------------------------
Null Hypothesis:
Sample was drawn from specified probability distribution
Alt. Hypothesis:
Sample was not drawn from specified probability distribution
Bins:                    10
Degrees of freedom:      9
Distribution:            ChiSquare(9)
Computed statistic:      15.8
Computed pvalue:         0.0711774
Critical value:          16.9189774487099
Result: [Accepted]
This statistical test does not provide enough evidence to conclude that the null hypothesis is false

 

(1)

# 2 parameters (mean and standard deviation) to fit from the sample S  INCORRECT ANSWER

ChiSquareSuitableModelTest(S, Normal(a, b), level = 0.5e-1, fittedparameters = 2):


print():
# verification
m := Mean(S);
s := StandardDeviation(S);
t := sqrt(add((S-~m)^~2) / (N-1));

print():
error "the estimation of the StandardDeviation ChiSquareSuitableModelTest is not correct";
print():

Chi-Square Test for Suitable Probability Model

----------------------------------------------
Null Hypothesis:
Sample was drawn from specified probability distribution
Alt. Hypothesis:
Sample was not drawn from specified probability distribution
Model specialization:    [a = -.2143e-1, b = .8489]
Bins:                    10
Degrees of freedom:      7
Distribution:            ChiSquare(7)
Computed statistic:      3.8
Computed pvalue:         0.802504
Critical value:          14.0671405764057
Result: [Accepted]
This statistical test does not provide enough evidence to conclude that the null hypothesis is false

 

 

HFloat(-0.021425681632689854)

 

HFloat(0.8531979363682092)

 

HFloat(0.8531979363682094)

 

 

Error, the estimation of the StandardDeviation ChiSquareSuitableModelTest is not correct

 

(2)

# ONLY 1 parameter (mean OR standard deviation ?) to fit from the sample S  STUPID ANSWER
#
# A stupid answer: the parameter to fit not being declared, the procedure should return
# an error of the type "don(t know what is the paramater tio fit"
ChiSquareSuitableModelTest(S, Normal(a, b), level = 0.5e-1, fittedparameters = 1):


print():
WARNING("ChiSquareSuitableModelTest should return it can't fit a single parameter");
print():

Chi-Square Test for Suitable Probability Model

----------------------------------------------
Null Hypothesis:
Sample was drawn from specified probability distribution
Alt. Hypothesis:
Sample was not drawn from specified probability distribution
Model specialization:    [a = -.2143e-1, b = .8489]
Bins:                    10
Degrees of freedom:      8
Distribution:            ChiSquare(8)
Computed statistic:      3.8
Computed pvalue:         0.874702
Critical value:          15.5073130558655
Result: [Accepted]
This statistical test does not provide enough evidence to conclude that the null hypothesis is false

 

 

Warning, ChiSquareSuitableModelTest should return it can't fit a single parameter

 

(3)

ChiSquareSuitableModelTest(S, Normal(a, 1), level = 0.5e-1, fittedparameters = 1):  #CORRECT ANSWER
print():

# verification
m := Mean(S);
print():

Chi-Square Test for Suitable Probability Model

----------------------------------------------
Null Hypothesis:
Sample was drawn from specified probability distribution
Alt. Hypothesis:
Sample was not drawn from specified probability distribution
Model specialization:    [a = -.2143e-1]
Bins:                    10
Degrees of freedom:      8
Distribution:            ChiSquare(8)
Computed statistic:      16.4
Computed pvalue:         0.0369999
Critical value:          15.5073130558655
Result: [Rejected]
This statistical test provides evidence that the null hypothesis is false

 

 

HFloat(-0.021425681632689854)

 

(4)

ChiSquareSuitableModelTest(S, Normal(0, b), level = 0.5e-1, fittedparameters = 1):  #CORRECT ANSWER

print():
# verification
s := sqrt((add(S^~2) - 0^2) / N);
print():

Chi-Square Test for Suitable Probability Model

----------------------------------------------
Null Hypothesis:
Sample was drawn from specified probability distribution
Alt. Hypothesis:
Sample was not drawn from specified probability distribution
Model specialization:    [b = .8492]
Bins:                    10
Degrees of freedom:      8
Distribution:            ChiSquare(8)
Computed statistic:      6.4
Computed pvalue:         0.60252
Critical value:          15.5073130558655
Result: [Accepted]
This statistical test does not provide enough evidence to conclude that the null hypothesis is false

 

 

HFloat(0.8491915633531496)

 

(5)

 


 

Download ChiSquareSuitableModelTest.mw

Please Wait...