::: {.content-visible when-format="typst"} ```{=typst} #set text(font: "New Computer Modern", size: 11pt) #set par(justify: true) ``` ::: **Time Allowed:** 120 minutes\ **Total Marks:** 50 ## Instructions {.unnumbered} This examination consists of two sections: - **Section A:** Multiple Choice Questions (10 questions × 3 marks = 30 marks) - **Section B:** Short Questions (2 questions × 10 marks = 20 marks) **Answer all questions.** Show all working for Section B questions. ::: {.content-visible when-format="typst"} ```{=typst} #line(length: 100%, stroke: 1pt) ``` ::: --- # Section A: Multiple Choice Questions (30 marks) *Select the best answer for each question.* ### Question 1 Which of the following best describes the purpose of k-fold cross-validation? a) To increase the size of the training dataset b) To estimate the test error of a statistical learning method c) To reduce the variance of predictions d) To eliminate the need for a validation set ### Question 2 In the bootstrap method, approximately what fraction of observations are NOT included in a given bootstrap sample? a) 1/n b) 1/e ≈ 0.368 c) 1/2 d) 1/3 ### Question 3 Ridge regression addresses multicollinearity by: a) Removing highly correlated predictors b) Setting some coefficients exactly to zero c) Shrinking all coefficients toward zero d) Orthogonalizing the predictor matrix ### Question 4 Which statement about Lasso regression is TRUE? a) It can only shrink coefficients, not set them to zero b) It performs variable selection by setting some coefficients to zero c) It requires all predictors to be on the same scale d) Both b and c are correct ### Question 5 In ridge regression, as the tuning parameter λ increases: a) Model flexibility increases and training RSS decreases b) Model flexibility decreases and training RSS increases c) Coefficient estimates approach their least squares values d) The bias-variance trade-off does not change ### Question 6 A natural cubic spline with K knots is represented by: a) K basis functions b) K + 1 basis functions c) K + 2 basis functions d) K + 4 basis functions ### Question 7 Which method would be most appropriate for modeling a non-linear relationship when you suspect different predictors have different functional forms? a) Polynomial regression b) Regression splines c) Generalized Additive Models (GAMs) d) Step functions ### Question 8 Bagging (Bootstrap Aggregation) reduces: a) Bias only b) Variance only c) Both bias and variance equally d) Neither bias nor variance ### Question 9 Random forests improve upon bagging by: a) Using a larger number of trees b) Randomly selecting a subset of predictors at each split c) Using deeper trees d) Pruning each tree more aggressively ### Question 10 Dropout is a regularization technique that: a) Removes less important input features b) Randomly sets a fraction of neurons to zero during training c) Reduces the number of hidden layers d) Decreases the learning rate over time # Section B: Short Questions (20 marks) ::: {.callout-note appearance="minimal"} ## Question 1 (10 marks) Suppose we have a dataset with n = 100 observations and p = 10 predictors. We want to compare three approaches: least squares regression, ridge regression, and lasso regression. **(a) (3 marks)** Explain conceptually why we might prefer ridge or lasso regression over least squares regression in this setting. **(b) (4 marks)** Describe a procedure to select the optimal tuning parameter λ for ridge regression using cross-validation. Include specific details about the steps involved. **(c) (3 marks)** Suppose after fitting lasso regression with the optimal λ, you find that only 4 of the 10 predictors have non-zero coefficients. How would you interpret this result? **Space for answer:** ::: {.content-visible when-format="typst"} ```{=typst} #v(12cm) ``` ::: ::: \ ::: {.callout-note appearance="minimal"} ## Question 2 (10 marks) You are building a classification model to predict customer churn (binary outcome) using a dataset with 20 predictor variables. You are considering using random forests and boosting. **(a) (4 marks)** Explain the key algorithmic difference between random forests and boosting. How does each method combine the predictions from multiple trees? **(b) (3 marks)** Both methods have tuning parameters that need to be selected. For random forests, identify two key tuning parameters and explain their role. For boosting, identify two key tuning parameters and explain their role. **(c) (3 marks)** After fitting both models, you find that the random forest has a test error rate of 12% and the boosting model has a test error rate of 10%, but the boosting model takes much longer to train. What trade-offs would you consider when choosing between these models? **Space for answer:** ::: {.content-visible when-format="typst"} ```{=typst} #v(12cm) ``` ::: ::: --- # Answer Key (For Instructor Use Only) ## Section A - Multiple Choice Answers 1. **b** - Cross-validation estimates test error 2. **b** - Approximately 1/e observations excluded 3. **c** - Ridge shrinks coefficients toward zero 4. **d** - Lasso performs variable selection and requires scaling 5. **b** - Higher λ → less flexibility, higher training RSS 6. **d** - Natural cubic spline with K knots has K + 4 basis functions 7. **c** - GAMs allow different forms for different predictors 8. **b** - Bagging reduces variance 9. **b** - Random forests randomly select predictors at each split 10. **b** - Dropout randomly zeros neurons during training ## Section B - Marking Guidelines ### Question 1 (10 marks) **Part (a) - 3 marks:** - Award marks for mentioning bias-variance trade-off (1 mark) - Handling multicollinearity (1 mark) - Feature selection for lasso or regularization benefits (1 mark) **Part (b) - 4 marks:** - Split data into k folds (1 mark) - Create grid of λ values (1 mark) - For each λ, compute k-fold CV error (1 mark) - Select λ with minimum CV error (1 mark) **Part (c) - 3 marks:** - Variable selection interpretation (1 mark) - Only 4 predictors are important (1 mark) - Model is more interpretable/parsimonious (1 mark) ### Question 2 (10 marks) **Part (a) - 4 marks:** - Random forests: parallel independent trees (1 mark) - RF combines via averaging/voting (1 mark) - Boosting: sequential trees (1 mark) - Boosting fits to residuals with weighted combination (1 mark) **Part (b) - 3 marks:** - RF parameters: B (number of trees), mtry (1.5 marks) - Boosting parameters: B, λ (learning rate), d (tree depth) (1.5 marks) **Part (c) - 3 marks:** - Discussion of accuracy vs computational cost (1 mark) - Production/real-time requirements (1 mark) - Model maintenance and interpretability (1 mark) ::: {.content-visible when-format="typst"} ```{=typst} #v(1cm) #align(center)[ #line(length: 50%, stroke: 2pt) #v(0.3cm) #text(12pt, weight: "bold")[END OF EXAMINATION] ] ``` ::: --- **END OF EXAMINATION**