Question 1

Unsupervised evaluation can be internal or external. Which of the following is an internal method for evaluating alternative clusterings produced by the K-Means algorithm?

Accepted Answer

A)   Use a production rule generator to compare the rule sets generated for each clustering.
B)   Compute and compare class resemblance scores for the clusters formed by each clustering.
C)   Compare the sum of squared error differences between instances and their corresponding cluster centers for each alternative clustering.
D)   Create and compare the decision trees determined by each alternative clustering. C

Question 2

The standard error is defined as the square root of this computation.

Accepted Answer

A)   The sample variance divided by the total number of sample instances.
B)   The population variance divided by the total number of sample instances.
C)   The sample variance divided by the sample mean.
D)   The population variance divided by the sample mean. A

Question 3

Data used to optimize the parameter settings of a supervised learner model.

Accepted Answer

A)   training
B)   test
C)   verification
D)   validation D

Question 4

Selecting data so as to assure that each class is properly represented in both the training and test set.

Accepted Answer

A)   cross validation
B)   stratification
C)   verification
D)   bootstrapping

Question 5

If a real-valued attribute is normally distributed, we know that approximately 95% of all attribute values lie within

Accepted Answer

A)   one standard deviation of the mean.
B)   two standard deviations of the mean.
C)   three standard deviations of the mean.
D)   four standard deviations of the mean.

Question 6

The average squared difference between classifier predicted output and actual output.

Accepted Answer

A)   mean squared error
B)   root mean squared error
C)   mean absolute error
D)   mean relative error

Question 7

The hypothesis of no significant difference.

Accepted Answer

A)   nil
B)   invalid
C)   null
D)   void

Question 8

We have performed a supervised classification on a dataset containing 100 test set instances. Eighty of the test set instances were correctly classified. The 95% test set accuracy confidence boundaries are:

Accepted Answer

A)   76% and 84%
B)   72% and 88%
C)   78% and 82%
D)   70% and 90%

Question 9

A decision tree is built to determine individuals likely to default on an unsecured loan. The null hypothesis states that an individual will not default on the loan. The decision tree correctly classifies 80% of the instances in a test dataset. Fifteen percent of the mistakes made by the model are type 1 errors. What can be said about the performance of the model?

Accepted Answer

A)   The accuracy of the model for correctly determining those individuals who did not default on their loan was at least 75%.
B)   The accuracy of the model for correctly determining those individuals who defaulted on their loan was at least 75%.
C)   The majority of errors made by the model accepted individuals who defaulted.
D)   The majority of errors made by the model rejected individuals who did not default.
E)   More than one of a,b,c or d is correct.

Question 10

The correlation coefficient for two real-valued attributes is -0.85. What does this value tell you?

Accepted Answer

A)   The attributes are not linearly related.
B)   As the value of one attribute increases the value of the second attribute also increases.
C)   As the value of one attribute decreases the value of the second attribute increases.
D)   The attributes show a curvilinear relationship.

Question 11

We have built and tested two supervised learner modelsF1F1F1S1F1F1F10M1 and M2. We compare the test set accuracy of the models using the classical hypothesis testing paradigm using a 95% confidence setting. 
The computed value of P is 2.53. What can we say about this result?

Accepted Answer

A)   Model M1 performs significantly better than M2.
B)   Model M2 performs significantly better than M1.
C)   Both models perform at the same level of accuracy.
D)   The models differ significantly in their performance.
E)   More than one of a,b,c or d is correct.

Question 12

Bootstrapping allows us to

Accepted Answer

A)   choose the same training instance several times.
B)   choose the same test set instance several times.
C)   build models with alternative subsets of the training data several times.
D)   test a model with alternative subsets of the test data several times.

Question 13

The correlation between the number of years an employee has worked for a company and the salary of the employee is 0.75. 
What can be said about employee salary and years worked?

Accepted Answer

A)   There is no relationship between salary and years worked.
B)   Individuals that have worked for the company the longest have higher salaries.
C)   Individuals that have worked for the company the longest have lower salaries.
D)   The majority of employees have been with the company a long time.
E)   The majority of employees have been with the company a short period of time.

Exam 7: Formal Evaluation Techniques