Blog | Karl S Study Blog

Karl 曹

Jun 6, 20227 min read

Comparison of Ridge Regression and OLS

This post will visualize how ridge regression outperforms OLS estimators. The figures and code of this post come from a group work in...

Karl 曹

Jun 4, 20224 min read

The Coefficients' Shrinkage Path of Ridge Regression

For ridge regression, it uses a shrinkage method by adding penalty on estimated coefficients. Usually ridge regression could deal with...

Karl 曹

May 21, 20227 min read

Compare LDA and Logistic Regression(2)

LDA assumes the distribution of x is normal, and it estimates the mean and variance by: and estimates the basic probability of class k...

Karl 曹

May 20, 20222 min read

Compare LDA and Logistic Regression(1)

Here we state the initial setting: We assume we have two classes, call them Class 1 and Class 2, obs. for class 1 is 300, and obs. for...

Karl 曹

May 17, 20228 min read

Compare the Confidence Intervals' Accuracy between OLS and Bootstrap

Initial setting: X belongs to a normal distribution, mean at 0 and select some variance sigma^2. Error term belongs to standard normal...

Karl 曹

May 16, 20221 min read

Two Types of Confidence Interval for Bootstrap

The regular precentile confidence interval for bootstrap result is easy, we just pick alpha/2 and 1 - alpha/2 quantile of estrimated...

Karl 曹

May 16, 20221 min read

Short Logistic Maximum Likelihood Function Derivation

Logistic maximum likelihood function(for binary response) in product form is not popular, here is a derivation from product form to sum...