Introduction to Data Science

Final Projects

Author

Joanna Bieri
DATA101

Important Information

Final Projects

The final project is your chance to show what you can do with data! You will pick a question you care about, find data that can help answer it, and then clean, explore, visualize, and explain what you find. The final project is 20% of your grade.

The best projects come from something you already love. Sports, music, biology, psychology, business, your hometown, a game you play… if you are curious about it, there is probably data about it.

Due Dates:

  • Tuesday 10/6 - Find some initial data for your Final Project (see Getting Started below)
  • Sunday 10/18 11:59pm - Data Check In due (individual work)
  • Sunday 11/8 11:59pm - Final Project Proposals due (individual work)
  • Tuesday 11/17 - Final Project Groups assigned, and we set up your group in class
  • Tuesday 12/1 and Thursday 12/3 - Group work days in class
  • Thursday 12/3 - Last day of class - Final Project Check In
  • Saturday 12/12 9:00am - Final Project Presentations (our final exam time). Final Projects and the Self Assessment are due at the same time.

Getting Started - Find Some Data (by 10/6)

Start looking for data now, while we are still learning the tools. Pick a topic you are curious about and look for a data set you can load into Python. Some good places to look:

A good data set for this class:

  • loads into a DataFrame with pd.read_csv() or pd.read_excel()
  • has at least a few hundred rows, and a mix of numerical and categorical variables
  • comes with an explanation of what each column means (sometimes called a data dictionary)
  • tells you where the data came from and who collected it

Make a folder called FinalProject in your repo and save your data there. NOTE - GitHub will not take a file bigger than 100 MB. If your data is that big, come talk to me.

Data Check In (individual work, due Sunday 10/18 11:59pm)

Make a notebook called DataCheckIn.ipynb in your FinalProject folder. In it:

  1. Say where your data came from (a link, and who collected it).
  2. Load your data into a DataFrame.
  3. Run .describe() on your data.
  4. Run .value_counts() on at least one categorical column.
  5. Make one plot.

Push it to your repo. This is a quick look, not a finished analysis! We will use this notebook again during Data Ethics week, so make sure your data really loads.

Project Proposals (individual work)

Your project proposal should reflect your own individual work and should not overlap too much with the work of your peers. Project proposals should take the form of a JupyterLab notebook that includes the following sections:

  • Introduction and Background for the problem you are interested in.
    • What is the question you hope you can answer?
    • Why is the question important to you?
    • What are you hoping to achieve with the project?
  • Description of the data.
    • Where did you get your data?
    • What data is found in your data set?
    • Are there any ethical concerns with using the data or how the data was obtained?
    • Is there a public location where the data is freely available? If so where? If not why?
  • Initial Exploratory Data Analysis
    • Definition of all variables (or explanation for why you are not considering some variables)
    • Descriptive statistics for the data set, counts of the variables, value counts (frequency tables) for each categorical variable of interest.
    • Plots that help a reader understand your data.
    • Initial analysis to answer initial data questions.
  • Proposal for further study.
    • What larger questions would you like to try to answer?
    • We start learning about models on Thursday 11/5. What do you think you might be able to predict with your data?
    • Are there any larger impacts that your study might have?

Push your proposal notebook to the FinalProject folder in your repo by Sunday 11/8 11:59pm. Whatever is in your repo at the deadline is what gets graded. If it’s pushed, it’s submitted.

Before you push, restart the kernel and run all the cells (Kernel > Restart Kernel and Run All Cells) to make sure your notebook runs from top to bottom. If it needs a data file, push the data file too, or read the data straight from a web address. Your notebook should contain just as many words and as much description of the problem/results as it does code.

If you already have a group of people you plan to work with - please list their names at the end of the proposal. I read all of the proposals and then put you into groups, usually around similar topics or data.

Final Project Groups

Groups should plan to use class time to meet and work together. We will spend Tuesday 11/17 in class getting your group set up.

After groups are assigned, groups will decide on a specific focused set of questions that they hope to answer and a data set (or data sets) that they want to focus on. Try to propose a nice narrative for your study that ties the different questions together.

Each group will work in one shared Google Colab notebook, so everyone can edit the same notebook. Have at least one organizational meeting where you divide up the parts of the work. Every person should contribute to the data analysis and writing portions of the work.

A few Colab tips:

  • Share the notebook with everyone in your group, and with me (joanna_bieri@redlands.edu).
  • Files you upload to Colab disappear when the session ends. Put your data in Google Drive, or read it straight from a web address.
  • Colab already has pandas and plotly installed.

Final Project Check In

Thursday 12/3 is the last day of class. Come to class ready to do a mini presentation of where your project is: your questions, your data, and a few results. Make sure I can see your Colab notebook and your presentation slides before class.

Final Projects Due

Final projects should be a carefully constructed notebook in your group’s Colab, shared with me. There should be the following sections:

  • Title, Authors and Abstract
  • Introduction and Background
  • Data Used
  • Exploratory Data Analysis
  • Proposed Questions
  • Analysis and Results
  • Conclusion

Before you push your final version, restart the kernel and run all the cells one more time to make sure everything works.

Presentations

Your group will present your project on Saturday 12/12 at 9:00am, our final exam time. Presentations are 10-15 minutes. You have lots of room to be creative with how you present, as long as we walk away knowing your question, what you found, and why it matters.

Self Assessment

Each person will fill out a short self and peer assessment on Canvas, due at the same time as the final project. You will tell me what you contributed, what you learned, and how each person in your group contributed. See Grading below.

AI Policy

The AI policy for the final project is the same as for homework. See the Artificial Intelligence section of the syllabus. The written discussion and interpretation in your project must be your own thinking, in your own words.

Grading

The final project is 20% of your course grade. Here is how the 100 project points break down:

Part Who Due Points
Data Check In individual Sunday 10/18 11:59pm 5
Project Proposal individual Sunday 11/8 11:59pm 15
Check In (mini presentation) group Thursday 12/3 in class 5
Final Notebook group Saturday 12/12 9:00am 40
Presentation group Saturday 12/12 9:00am 20
Self and Peer Assessment individual Saturday 12/12 9:00am 15

For most rows below: Full means all the points, Partial means about half, and Missing means zero.

Data Check In (5 pts) - graded on completion. If your notebook is pushed and has all five pieces (source, data loaded, .describe(), .value_counts(), one plot) you get all 5 points.

Project Proposal (15 pts)

Full Partial Missing
Question and background (3) Clear question, and why it matters to you A topic but no real question Missing
Data description (3) Where it came from, who collected it, what the variables are Some of these Missing
Ethics (2) At least one real ethical concern with this data (or why there isn’t one) “No concerns” with no reasoning Missing
Exploratory analysis (4) Descriptive statistics, value counts, and plots, each with a sentence saying what it shows Output with little or no explanation Missing
Further study (2) Two or three bigger questions to explore Vague Missing
Runs top to bottom (1) Restart and Run All works Errors

Check In (5 pts)

Full Partial Missing
Mini presentation (3) Shows your questions, your data, and a few results Shows some progress Not there
Draft shared (2) Colab notebook and slides shared with me before class Only one of the two Neither

Final Notebook (40 pts)

Full Partial Missing
Abstract, introduction, and questions (6) A reader knows what you asked and why before seeing any code Questions are there but hard to find Missing
Data (6) Source, how it was collected, variables, and any cleaning you did, with reasons Cleaning happens with no explanation Missing
Exploratory analysis and visuals (8) Plots have titles, labeled axes, and a sentence saying what to notice Default labels or no explanation Missing
Analysis and results (10) Each question gets code, output, and a written answer Output without interpretation, or questions left unanswered Missing
Conclusion and limitations (4) What you found, what your analysis can’t tell you, what you would do next Summary only Missing
Ethics (3) Ethical concerns with your data or analysis, specific to your project A generic statement Missing
Runs top to bottom (3) Restart and Run All works in Colab Runs after small fixes Does not run

Presentation (20 pts) - be creative! I am grading what we walk away with, not the format.

Full Partial
Story (6) We learn your question, what you found, and why it matters Results without a clear question or takeaway
Visuals (6) Graphs are readable from the back of the room and each one has a point Hard to read, or too many
Everyone takes part (4) Every group member presents or clearly contributes Some members
Time (2) 10-15 minutes Under 8 or over 18 minutes
Questions (2) You can answer questions about your data and your choices Some trouble

Self and Peer Assessment (15 pts) - a short Canvas quiz.

  • Self assessment (5 pts, completion): what you contributed (be specific, eg. “I cleaned the date column and made the three maps in the results section”), one thing you learned, what you would do differently, and whether you used AI and what for.
  • Peer rating (10 pts): you rate each person in your group, and they rate you. Your points come from what your group members say about you:
    • Did their share or more: 10 points
    • Did less than their share: 5 points
    • Did not contribute: 0 points

Your peer ratings are private. Only I see them.

Final Project Outline

Below are some .ipynb cells that outline how you might write up your final project.

import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import plotly.express as px
from plotly.subplots import make_subplots
import plotly.io as pio
pio.renderers.default = 'notebook_connected'

Title: FINAL PROJECT OUTLINE

By: Author Names

Abstract

Here you type a 3-4 sentence summary of your final project.

Introduction and Background

Convince your audience that the rest of the project is worth learning about. What interesting problem are you exploring? Why should EVERYONE be interested in your analysis?

Data Used

Give some background about your data. Where did you get it? How was it compiled? How many observations? What are the variables?

Show a data frame with the most important columns.

Do some descriptive statistics and talk about the results.

## This section will have some code. Make sure to use Markdown cells to explain what the code is demonstrating

Exploratory Data Analysis

Show a few plots or some data tables that help your reader understand your data better. What are some initial questions that you were able to quickly answer? How do those questions lead you to a deeper analysis?

## This section will have some code. Make sure to use Markdown cells to explain what the code is demonstrating

Proposed Questions

What specific questions are you going to explore and present in the rest of the paper? Give a very brief overview of what you are going to do to answer those questions. Imagine this as a road map so your reader knows what to expect.

Analysis and Results

Here is where you show your code and results (plots, tables, predictions, etc) that help to explore and answer your questions. There should be code here that can be run to reproduce your results and conclusions. After each plot/table/final number, you should add a Markdown cell where you explain to the reader what the result means and how/why it answers the question.

## This section will have Lots of code. Make sure to use Markdown cells to explain what the code is demonstrating

Conclusion

Give a brief statement about what you achieved in your analysis, what issues or limitations your analysis contains, a conversation of any ethical concerns with the data or analysis, and some possible future directions (if you had more time and money to keep going).