Intermediate Data Science

Final Projects

Author

Joanna Bieri
DATA201

Important Information

Final Projects

The final project is your chance to show what you can do with the tools from this class, on a question that YOU care about. It is 20% of your course grade.

You start on your own: you find data and write a proposal by yourself. After Exam 2 you will be put in a team of 2-4 students (your third team of the semester), and the team builds one final project together.

How to hand things in: individual work (the Draft Proposal and the Project Proposal and Exploratory Data Analysis) is submitted on Canvas AND pushed to a FinalProject folder in your sandbox, the same way you handed in your exam. Team work happens on git, in your team’s project repo.

Due Dates:

  • Sunday 10/25 11:59pm - Draft Proposal (individual work, on Canvas and in your sandbox)
  • Tuesday 11/10 11:59pm - Project Proposal and Exploratory Data Analysis (individual work, on Canvas and in your sandbox)
  • Tuesday 11/24 - Final Project teams announced, after Exam 2
  • Saturday 12/12 at Noon - Final Projects due (one notebook per team, pushed to your team’s project repo) and Final Project Presentations (our final exam time)

Getting Started - Find Some Data

Start looking for data now! Pick something you are curious about (eg. your major, a sport, music, a job you would like to have, your home town). If you are curious about it, there is probably data about it. Here are a few places to look:

Good data for this project has at least a few hundred observations, a mix of numerical and categorical variables, and something you could try to predict. After Fall Break we start building models, so think about what you might want to predict with your data.

Draft Proposal (individual work, due Sunday 10/25 11:59pm)

This does not have to be formal! I just want to see that you are thinking about your project and that you have started looking for data. Write a few sentences for each of these. Submit it on Canvas and push it to the FinalProject folder in your sandbox:

  1. Your idea. What topic are you interested in? What is a question you would like to answer?
  2. Your data. Have you found data? Where is it from (give the link)? If you have not found data yet, tell me where you have looked and what you are hoping to find.
  3. A first look. If you have data, load it and tell me how many observations and variables it has. A screenshot or a few lines of output from .head() and .shape is plenty.
  4. Worries. Is there anything you are stuck on or worried about?

It is completely fine if your idea changes after this. The goal is to get you started early so that the Project Proposal and Exploratory Data Analysis is not a surprise.

Project Proposal and Exploratory Data Analysis (individual work, due Tuesday 11/10 11:59pm)

Your Project Proposal and Exploratory Data Analysis should reflect your own individual work and should not overlap too much with the work of your peers. It should take the form of a professionally written JupyterLab notebook that includes the following sections:

  • Introduction and Background for the problem you are interested in.
    • What is the question you hope you can answer?
    • Why is the question important to you?
    • What are you hoping to achieve with the project?
  • Description of the data.
    • Where did you get your data?
    • What data is found in your data set?
    • Are there any ethical concerns with using the data or how the data was obtained?
    • Is there a public location where the data is freely available? If so where? If not why?
  • Initial Exploratory Data Analysis
    • Definition of all variables (or explanation for why you are not considering some variables)
    • Descriptive statistics for the data set, counts of the variables, value counts (frequency tables) for each categorical variable of interest.
    • Plots that help a reader understand your data.
    • Initial analysis to answer initial data questions.
  • (optional) Initial Modeling
    • Have you tried any simple models to answer any deeper questions? If so, what have you tried?
  • Proposal for further study.
    • What larger questions would you like to try to answer?
    • What predictive analysis do you think might be possible?
    • Are there any larger impacts that your study might have?

Submit your notebook as an .ipynb file on Canvas AND push it to the FinalProject folder in your sandbox. Make sure your notebook can be run completely. If it requires a data file make sure to submit that file too. It should contain just as many words and as much description of the problem/results as it does code.

If you already have people you would like to work with, please list their names at the end of the proposal.

Final Project Teams

Final projects are done in teams of 2-4 students. I will announce the teams on Tuesday 11/24, after Exam 2, using your proposals so that people with similar interests can work together.

Once you have your team:

  • Decide on a specific focused set of questions that you hope to answer and the data set (or data sets) that you want to use. You can pick one person’s proposal or combine ideas. Try to propose a nice narrative for your study that ties the different questions together.
  • All of your team’s work happens on git, in a team repo for the project. Work the same way you have for homework all semester: your own branch, Pull Requests, and a teammate reviews before merging.
  • Have at least one organizational meeting where you divide up the parts of the work. Every person should contribute to the data analysis AND the writing.

Final Projects Due (Saturday 12/12 at Noon)

Final projects should be a carefully constructed JupyterLab Notebook, one per team, pushed to your team’s project repo before presentations start. Whatever is merged into main at Noon is what gets graded. There should be the following sections:

  • Title, Authors and Abstract
  • Introduction and Background
  • Data Used
  • Exploratory Data Analysis
  • Proposed Questions
  • Analysis and Results
  • Conclusion

This project should represent the best you can do given the tools that you learned in class. Every project will be different based on the data used. You must have clearly written words describing your decisions and conclusions. All of your conclusions must be supported by your code and the data.

Important requirements:

  • Your code must run from start to finish without errors.
  • Your project should load the data directly from an online source, or the data file must be in your team’s repo.
  • Include at least one machine learning method from class (eg. decision trees, KNN, regularized regression, Naive Bayes, dimensionality reduction).

Presentations (Saturday 12/12 at Noon)

Each team presents their results during our final exam period. Tell us your question, show us your data, and walk us through what you found. Everyone on the team should speak.

Final Project Outline

Below are some .ipynb cells that outline how you might write up your final project.

FINAL PROJECT TITLE

By: Your Name(s)

Abstract

Write a concise 3-4 sentence summary of: - The problem you investigated
- The dataset you used
- The methods applied
- Your main findings

Introduction and Background

Provide context for your project:

  • What motivated this topic?
  • Why should someone care about the problem you’re addressing?
  • What prior work, domain knowledge, or assumptions matter?
  • What is the goal of the analysis?

Your job is to convince the reader that your investigation is meaningful.

Data Used

Describe your dataset:

  • Source and method of collection
  • Number of observations and variables
  • Key fields and their meanings
  • Whether the data is real or synthetic
  • Ethical concerns (privacy, bias, representativeness)

Below you will preview the dataset, compute descriptive statistics, and summarize what you learn.

Notes from Data Summary

Explain what the descriptive statistics and value counts reveal about: - Trends - Outliers - Missing values - Data quality issues

Exploratory Data Analysis (EDA)

In this section include a small set of the most informative visualizations or tables, such as:

  • Distributions of key variables
  • Relationships between variables
  • Correlations
  • Missing-data patterns
  • Scatter or Pairplots

Do not include every plot you generated, choose only those that genuinely help tell the story.

Explain what early insights you gained and how they shaped your later analysis.

Interpretation of EDA

Explain what the plots/tables show and how they informed your next steps.

Proposed Questions

List the specific questions your project investigates.

Good guiding questions: - Are clear and measurable
- Can be answered with your data
- Are linked to your motivation
- Suggest reasonable analysis or modeling approaches

Provide a brief plan for how you will address each question.

Analysis and Results

This is the core of the project.

For each research question:

  1. Show the code you used (clean, runnable, documented).
  2. Include outputs such as tables, plots, metrics, or model results.
  3. Immediately follow each output with a Markdown explanation of:
    • What the result means
    • Why it matters
    • Whether it answers the question

Include at least one advanced machine learning method from class (e.g., decision trees, KNN, regularized regression, Naive Bayes, Dimensionality Reduction).

Be sure to: - Split into train/test - Scale the data - Avoid data leakage
- Justify your choices
- Evaluate your models carefully

Interpretation of Results

Explain what the model results show: - Do they answer the research question? - How strong is the model? - What limitations or caveats exist?

Conclusion

Summarize the project:

  • What did you accomplish?
  • What did the analysis reveal?
  • What limitations should the reader keep in mind?
  • What ethical considerations (bias, fairness, privacy) matter?
  • What future work would you pursue with more time or resources?

This section should tie the entire narrative together.