Lecture 9 - Exercise¶
In [ ]:
import numpy as np
import seaborn as sns
import pandas as pd
import matplotlib.pyplot as plt
In [ ]:
hob = pd.read_csv("https://facultyweb.cs.wwu.edu/~wehrwes/courses/data311_25f/lectures/L09/hobbies.csv", header=None)
hob
Your task: use Spacy's lemmatization functionality to see if you can find common hobbies amongst the survey responses.
In [1]:
# one-time setup:
!python -m spacy download en_core_web_sm
!pip install spacytextblob
Defaulting to user installation because normal site-packages is not writeable
Collecting en-core-web-sm==3.8.0
Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl (12.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 45.3 MB/s 0:00:00m0:00:01
✔ Download and installation successful
You can now load the package via spacy.load('en_core_web_sm')
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: spacytextblob in /cluster/home/wehrwes/.local/lib/python3.12/site-packages (5.0.0)
Requirement already satisfied: spacy>=3.0.0 in /opt/miniforge/lib/python3.12/site-packages (from spacytextblob) (3.8.7)
Requirement already satisfied: textblob>=0.18.0.post0 in /cluster/home/wehrwes/.local/lib/python3.12/site-packages (from spacytextblob) (0.19.0)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (1.0.13)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (2.0.11)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (3.0.9)
Requirement already satisfied: thinc<8.4.0,>=8.3.4 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (8.3.6)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (1.1.3)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (2.5.1)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (2.0.10)
Requirement already satisfied: weasel<0.5.0,>=0.1.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (0.4.1)
Requirement already satisfied: typer<1.0.0,>=0.3.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (0.19.2)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (4.67.1)
Requirement already satisfied: numpy>=1.19.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (2.3.3)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (2.32.5)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (2.11.9)
Requirement already satisfied: jinja2 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (3.1.6)
Requirement already satisfied: setuptools in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (80.9.0)
Requirement already satisfied: packaging>=20.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (25.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /opt/miniforge/lib/python3.12/site-packages (from spacy>=3.0.0->spacytextblob) (3.4.1)
Requirement already satisfied: language-data>=1.2 in /opt/miniforge/lib/python3.12/site-packages (from langcodes<4.0.0,>=3.2.0->spacy>=3.0.0->spacytextblob) (1.3.0)
Requirement already satisfied: annotated-types>=0.6.0 in /opt/miniforge/lib/python3.12/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy>=3.0.0->spacytextblob) (0.7.0)
Requirement already satisfied: pydantic-core==2.33.2 in /opt/miniforge/lib/python3.12/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy>=3.0.0->spacytextblob) (2.33.2)
Requirement already satisfied: typing-extensions>=4.12.2 in /opt/miniforge/lib/python3.12/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy>=3.0.0->spacytextblob) (4.15.0)
Requirement already satisfied: typing-inspection>=0.4.0 in /opt/miniforge/lib/python3.12/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy>=3.0.0->spacytextblob) (0.4.1)
Requirement already satisfied: charset_normalizer<4,>=2 in /opt/miniforge/lib/python3.12/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->spacytextblob) (3.4.3)
Requirement already satisfied: idna<4,>=2.5 in /opt/miniforge/lib/python3.12/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->spacytextblob) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/miniforge/lib/python3.12/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->spacytextblob) (2.5.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/miniforge/lib/python3.12/site-packages (from requests<3.0.0,>=2.13.0->spacy>=3.0.0->spacytextblob) (2025.8.3)
Requirement already satisfied: cloudpickle>=2.2.0 in /opt/miniforge/lib/python3.12/site-packages (from srsly<3.0.0,>=2.4.3->spacy>=3.0.0->spacytextblob) (3.1.1)
Requirement already satisfied: ujson>=1.35 in /opt/miniforge/lib/python3.12/site-packages (from srsly<3.0.0,>=2.4.3->spacy>=3.0.0->spacytextblob) (5.11.0)
Requirement already satisfied: blis<1.4.0,>=1.3.0 in /opt/miniforge/lib/python3.12/site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.0.0->spacytextblob) (1.3.0)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /opt/miniforge/lib/python3.12/site-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.0.0->spacytextblob) (0.1.5)
Requirement already satisfied: click>=8.0.0 in /cluster/home/wehrwes/.local/lib/python3.12/site-packages (from typer<1.0.0,>=0.3.0->spacy>=3.0.0->spacytextblob) (8.3.0)
Requirement already satisfied: shellingham>=1.3.0 in /opt/miniforge/lib/python3.12/site-packages (from typer<1.0.0,>=0.3.0->spacy>=3.0.0->spacytextblob) (1.5.4)
Requirement already satisfied: rich>=10.11.0 in /opt/miniforge/lib/python3.12/site-packages (from typer<1.0.0,>=0.3.0->spacy>=3.0.0->spacytextblob) (14.1.0)
Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in /opt/miniforge/lib/python3.12/site-packages (from weasel<0.5.0,>=0.1.0->spacy>=3.0.0->spacytextblob) (0.22.0)
Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in /opt/miniforge/lib/python3.12/site-packages (from weasel<0.5.0,>=0.1.0->spacy>=3.0.0->spacytextblob) (7.3.1)
Requirement already satisfied: wrapt in /opt/miniforge/lib/python3.12/site-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.1.0->spacy>=3.0.0->spacytextblob) (1.17.3)
Requirement already satisfied: marisa-trie>=1.1.0 in /opt/miniforge/lib/python3.12/site-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy>=3.0.0->spacytextblob) (1.2.1)
Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/miniforge/lib/python3.12/site-packages (from rich>=10.11.0->typer<1.0.0,>=0.3.0->spacy>=3.0.0->spacytextblob) (4.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/miniforge/lib/python3.12/site-packages (from rich>=10.11.0->typer<1.0.0,>=0.3.0->spacy>=3.0.0->spacytextblob) (2.19.2)
Requirement already satisfied: mdurl~=0.1 in /opt/miniforge/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->typer<1.0.0,>=0.3.0->spacy>=3.0.0->spacytextblob) (0.1.2)
Requirement already satisfied: nltk>=3.9 in /cluster/home/wehrwes/.local/lib/python3.12/site-packages (from textblob>=0.18.0.post0->spacytextblob) (3.9.1)
Requirement already satisfied: joblib in /opt/miniforge/lib/python3.12/site-packages (from nltk>=3.9->textblob>=0.18.0.post0->spacytextblob) (1.5.2)
Requirement already satisfied: regex>=2021.8.3 in /cluster/home/wehrwes/.local/lib/python3.12/site-packages (from nltk>=3.9->textblob>=0.18.0.post0->spacytextblob) (2025.9.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/miniforge/lib/python3.12/site-packages (from jinja2->spacy>=3.0.0->spacytextblob) (3.0.2)