This repository contains the materials for D-Lab’s DH Text Analysis workshop. Basic experience with Python (e.g. through Python Fundamentals) is expected.
This workshop series introduces several approaches to text analysis in the Digital Humanities.
After completing the notebooks in this repo, you will be able to:
- Understand the need for tokenization when computationally analyzing textual corpora.
- Engage with NLTK to explore textual corpora.
- Use TF-IDF to compare the writing styles of authors.
- Use topic modeling to find themes and topics in a corpus.
- Use Named Entity Recognition to trace persons, times, locations in a corpus.
- Understand how TEI XML-encoded corpora allow you to ask more advanced questions.
- Understand how to finetune a GPT2 model on literary corpora.
This workshop does not cover the following:
- Programming concepts such as for-loops, if-statements, and writing your own functions. These are covered in Python Intermediate.
- Advanced DataFrame manipulation. This is covered in Python Data Wrangling.
- Data visualization. This is covered in Python Intermediate and Python Data Visualization.
Before attending the workshop, you should install Python and Jupyter to your computer. If you need help, please submit a consulting request with D-Lab prior to the start of the workshop.
Anaconda is software that allows you to run Python and Jupyter notebooks on your computer. Installing Anaconda is the easiest way to make sure you have all the necessary software to run the materials for this workshop. Complete the following steps:
- Download and install Anaconda (Python 3.9 distribution). Click "Download" and then click 64-bit "Graphical Installer" for your current operating system.
- Download the materials in this repository:
- Click the green "Code" button in the top right of the repository information.
- Click "Download Zip".
- Extract this file to a folder on your computer where you can easily access it (we recommend Desktop).
- Optional: if you're familiar with
git, you can instead clone this repository by opening a terminal and enteringgit clone [email protected]:dlab-berkeley/Python-Fundamentals-I.git.
Now that you have all the required software and materials, you need to run the code:
-
Open the Anaconda Navigator application. You should see the green snake logo appear on your screen. Note that this can take a few minutes to load up the first time.
-
Click the "Launch" button under "Jupyter Lab" and navigate through your file system to the
Python-Fundamentalsfolder you downloaded above. -
Navigate to lessons -> Fundamentals-I
-
Open the
01_Jupyter_and_Python.ipynbto begin. -
Press Shift + Enter (or Ctrl + Enter) to run a cell.
Note: if you are having trouble accessing the Downloads/Desktop folder from Anaconda Navigator, check out this post or submit a consulting request to get help!
If you do not have Anaconda installed and the materials loaded on your workshop by the time it starts, we strongly recommend using the UC Berkeley Datahub to run the materials for these lessons. You can access the DataHub by clicking this button:
The DataHub downloads this repository, along with any necessary packages, and allows you to run the materials in a Jupyter notebook that is stored on UC Berkeley's servers. No installation is necessary from your end - you only need an internet browser and a CalNet ID to log in. By using the DataHub, you can save your work and come back to it at any time. When you want to return to your saved work, just go straight to DataHub, sign in, and you click on the Python-Fundamentals folder.
D-Lab works with Berkeley faculty, research staff, and students to advance data-intensive social science and humanities research. Our goal at D-Lab is to provide practical training, staff support, resources, and space to enable you to use R for your own research applications. Our services cater to all skill levels and no programming, statistical, or computer science backgrounds are necessary. We offer these services in the form of workshops, one-to-one consulting, and working groups that cover a variety of research topics, digital tools, and programming languages.
Visit the D-Lab homepage to learn more about us. You can view our calendar for upcoming events, learn about how to utilize our consulting and data services, and check out upcoming workshops.
Here are other Python workshops offered by the D-Lab:
- Computational Text Analysis in Python
- Introduction to Machine Learning in Python
- Fairness and Bias in Machine Learning
- Tom van Nuenen