DATA 311 - Lab 6: Measuring Bias in Language Models

Collaboration Policy

This lab is to be completed individually. You may (are encouraged) to spend the lab period working together with a partner. After the lab period ends, you will work independently and submit your own solution, though you may continue to collaborate in accordance with the individual assignment collaboration policy listed on the syllabus.

Your Tasks

The details for Lab 6 are provided inline in the Lab 6 starter notebook. Please read through the notebook and complete the tasks labeled TODO 1-9.

Submission

Make sure that all your cells have been run and have the output you intended, then download a copy of your notebook in .ipynb format and submit it to Canvas.

Survey

Finally, fill out the Lab 6 Survey on Canvas. Your submission will not be considered complete until you have submitted the survey.

Rubric

This lab has 9 TODOs which together are worth a total of 45 points.

• (4 points) TODO 1: Build a DataFrame with the BERT predictions
• (4 points) TODO 2: Write a function to identify results with punctuation or stopwords
• (4 points) TODO 3: Filter results for punctuation and stopwords
• (4 points) TODO 4: Apply sentiment analysis to all the predicted sentences.
• (4 points) TODO 5: Calculate summary statistics of polarity per dataset.
• (10 points) TODO 6: Make a nice plot showing the sentiment analysis results per dataset.
• (5 points) TODO 7: Find the most commonly-occurring predictions per dataset.
• (8 points) TODO 8: Discussion
• (2 points) TODO 9: Reflection

TODOs 1-5 and 7 are graded according to the usual Correct, Convincing, and Clear criteria.

TODO 6 is graded on the same criteria as your plots from Lab 3.

TODOs 8 and 9 are graded for insight and clarity.

Extra Credit

Up to 3 points as detailed in the notebook.