Rain Prediction
A final project for Northeastern University's DS4400 Machine Learning and Data Mining Class. Using Kaggle's Australia Rain Dataset, we will hand write the code for different machine learning models in order to better learn the applications of, in our case, classification models and enhance our coding techniques when it comes to Machine Learning.
The setup
For our project, we were trying to implement several ML models that will predict whether or not it will rain a given day, due to various features describing the weather the day before. The problem can be described as a binary classification problem with the two classes being that “it will rain”/ “it will not rain”. Given that weather can greatly affect one’s plans for the day, such as whether they will spend the day outside, what they will wear, etc, this can prove to be helpful in planning purposes for many. Additionally, with the topic of climate change being one of great importance, tracking changes in rainfall prediction could prove to be an investigation of great value. We utilized the Kaggle Australia Rain Dataset.
Results
Logistic Regression
Accuracy 0.50396
F1 0.670177
Precision 0.50396
Recall 1.0
Random Forest (Subset of 20) (x3)
Accuracy: 0.55/53/52
F1: 0.30/48/63
Precision: 0.71/54/51
Recall: 0.19/44/81
Naive Bayes
Accuracy: 0.741081
F1: 0.733452
Precision: 0.762120
Recall: 0.706862
SVM
Accuracy: 0.773387
F1: 0.768948
Precision: 0.790824
Recall: 0.748250
For the models, we looked at accuracy, precision, recall and the F1 score regarding the prediction for whether it will rain tomorrow. We found consistent 'no' guessing by the logistic regression model. Naive Bayes performed better with a .74 ration of correct predictions. SVM appeared to have the best metrics so far, with all scores falling between .74 and .791. Our Random Forest was most interesting, taking some struggling, and ended up running results multiple times on a smaller subset. It was easy to see random guessing with logistic regression and time complexity with random forest.
Reflections
This project has taught us many lessons about the ways that we build models, and the assumptions we work with when it comes to choosing both pretrained models and those we write ourselves. It was very challenging to write the models by hand- it required utilization of mathematical concepts and understanding how our transformations of the data created a path or algorithm a computer could use to ‘find’ the answer, or in this case class, to our problem. We also had to take time to understand how to communicate data and insights in a way each other and graders could understand it. This was a valuable way to interact with the class and its goals, but it’s an understanding that can extend past the timeline of the project.
As far as future work is concerned, our work on this project taught us a lot about the creation of models and looking at the data we work with before using it. Given a longer deadline, our group would work to improve our model coding in efficiency. We wish also to delve further into the ways correlation affects data. It was obvious a lot of different factors, such as the wind changes throughout the day, were often correlated with each other. But, what if we instead recorded the positive and negative change maybe throughout the day for wind gusts? Taking a piece by piece approach to this data could give plenty of individual insight into the small-scale particulars this weather data covers and might be able to make us better at handling data and models in the future.
