B. Regression Models

Welcome to Airbnb Analysis Corp.! Your task is to set the competitive daily accommodation rate for a client's house in Bondi Beach. The owner currently charges $500. We have been tasked to estimate a fair value that the owner should be charging. The house has the following characteristics and constraints. While developing this model you came to realize that Airbnb can use your model to estimate the fair value of any property on their database, your are effectively creating a recommendation model for all prospective hosts!

The problem with the traditional OLS regression method is that the target outcome $y$ given the features does not always follow a Gaussian distribution.

Classic polynomial regression is trivially implemented by piping polynomial features with linear regression in scikit-learn.

Any and all transformation or operations can potentially be used to improve the dataset.

Quite naturally, using all of these before-mentioned augmentation methods, you can create a dataset that is infinitely large with an infinite amount of features.

However, multiple methods have been developed to automatically create models that are simplified by adding a penalty term to an augmentation method.

So how is this different to Ridge and LASSO regressions?

GAM is a linear model with a key difference when compared to ordinary Generalised Linear Models.

Weighted linear regression is a generalization of linear regression where the covariance matrix of errors is incorporated in the model.

The method of ordinary least squares assumes that there is constant variance in the errors (which is called homoscedasticity).

The method of weighted least squares can be used when the ordinary least squares assumption of constant variance in the errors is violated (which is called heteroscedasticity).

where $\epsilon^{*}$ is assumed to be (multivariate) normally distributed with mean vector 0 and nonconstant variance-covariance matrix

If we define the reciprocal of each variance, $\sigma_{i}^{2}$, as the weight, $w_{i}=1 / \sigma_{i}^{2}$ then let matrix W be a diagonal matrix containing these weights:

KNN (k-nearest neighbors) is a simple algorithm, there's no "learning" algorithm as such— you just store the training data. Then, to predict a new input $x$, you statistically find the $K$ closest points in the training data, and average the corresponding outputs. We can write this as an equation as

Here the function $\text{Neigh}_K(x)$ returns the indices of the $K$ closest points in the training data. This is easy to understand through an example. Imagine we have the following one-dimensional dataset with five points.

Let's compute the following neighborhoods. In each case, as the answer to $\text{Neigh}_K(x)$, we expect a set of $K$ indices of points which are closest neighbors of $x$ in our data.

For example, $\mathrm{Neigh}_2(1)=\{1,4\}$ because the two closest points to $x=1$ are $x^{(1)}=1$ and $x^{(4)}=2$. Similarly, $\mathrm{Neigh}_2(9.5)=\{2,3\}$ since the two closest points to $x=9.5$ are $x^{(2)}=9$ and $x^{(3)}=10$.

Now, let's now discuss a two-dimensional dataset. Consider the following dataset with two input dimensions $x$ and one output $y$.

Some researchers like to start by identifying the different components that determines the evolution of phenomena over time, such as trends, cycles, seasonality, and random disturbance.

The purpose here is to get a feel for what the time series look likes, data scientists would view this decomposition as part of Exploratory Data Analysis (EDA).

For example let’s say that we are tracking the the total revenue produced by all publicly listed companies on US exchanges.

Normal decision trees are often used within a financial context, especially when considering capital allocation problems, and often appears as a topic in corporate finance papers.

For example let’s consider the following problem.

We can now use this decision tree to identify the expected value or net present value, $EV_{expand}=0.5\times5+0.5\times-3$ and $EV_{nothing}=0.5\times2+0.5\times-1$, as such $EV_{expand}>EV_{nothing}$ or $2m > $1m.

For this homework assignment, we have three different notebooks, your job is to write one sentence for each notebook discribing:

You can present your answers however you please, and you don’t have to run experiments or submit any code, you can simply provide a textual response. For example, if you wanted to present it in table format:

When oil exits a bear market, the currency of oil-producing nations should also rebound. With this strategy, we will investigate the effect the price of oil has on the Norwegian krone (NOK) and identify whether a profitable trading strategy can be executed.

To start we need a stabilizer currency to regress against. The currency should be unrelated to the currency under investigation. Something like the Japanese yen ( JPY) is a good candidate. From here on, one would use the price of the NOK and Brent as measured against JPY to identify whether the Norwegian currency is under or overvalued.

We will use an elastic net regression as the machine-learning technique. It is a good tool when multicollinearity is an issue. You haven’t learned much about elastic net regressions, your homework is to understand them and to see if you can understand the strategy and summarize it in a two paragraphs.

An elastic net is a regularized regression method that combines both L1 (LASSO) and L2 (Ridge) penalties. The estimates from the elastic net method are defined by:

The loss function becomes strongly convex as a result of the quadratic penalty term, therefore providing a unique minimum. Now that the predictors are in place, one has to set up a pricing signal; one sigma two-sided is the common practice in arbitrage. We short if it spikes above the upper threshold and long on the lower threshold.

The stop-loss will be set at two standard deviations. At that point, one can expect the interpretation of the underlying model to be wrong and therefore choose to exit the position.