DIGIT 210: Large Scale Text Analysis

• Join our DIGIT Slack group (See Canvas announcement.)
• Get the big picture: Read the whole Section 1: Basics (all the short sections) in the XML FAQ about what XML is even for.

• Complete XML Exercise 1

• Do you have a red square in oXygen on your first XML? Address any issues with well-formedness.
• Try adding tags and attributes to help track more specific pieces of information than you marked at first, or try to tag a little differently to organize information a little more efficiently, keeping related things tagged the same way, separating numbers from units, etc.
• Resubmit code for XML Ex 1 if I asked you to revise and resubmit (if it was not well-formed). Otherwise, just submit your revisions as part of XML Exercise 2 on Canvas
• Now, choose another recipe with at least five ingredients and multiple steps, to code. Try to code this recipe in a way that is consistent with the bread recipe you just coded: try to track the same kinds of information using the same elements and attributes. Submit your newly coded files (both of them) on Canvas for XML Exercise 2.

• Read the git manual 1.1: "Getting Started—About Version Control"
• Watch my video introducing the Bash shell and "home" on your computers and practice the shell commands I am introducing.

• Complete Git Exercise 1 and Git Exercise 2.
• (Optional: good to try:): Watch / work with Set up a Git Bash Profile (short video how-to): (this is convenient for you, and more command line knowledge).

• XML Exercise 3: Review our feedback on your coding exercises so far and submit revisions if we asked you to. Mark up a text of your choice (any genre; manageable but reasonable size; non-English languages welcome): With your markup:
  • Make a hierarchy at least three or four levels deep (it can be deeper!)
  • Work on applying attributes with your elements, and doing so in a careful and systematic way to make specific information easy to retrieve from your code.
  Work on applying attributes with your elements, and doing so in a careful and systematic way.
• Seven Days of Git: Day 2 of 7

• Watch ~20+ min. video: Introducing Regular Expressions in the oXygen XML Editor
• Regex Exercise 1. As you work on this, consult our Intro to Regular Expressions and the Regular Expressions Quick Start.
• Seven Days of Git: Day 4 of 7 (We will count the weekend as one day.)

• Regex Exercise 2: Sonnets
• Seven Days of Git: Day 5 of 7

• Discussion of professional and student DH projects: first post
• Regex Exercise 3: Blithedale novel
• Seven Days of Git: Day 6 of 7

• Discussion of professional and student DH projects: second post
• Regex Exercise 4: Mulan Screenplay
• Revisions of problematic regex assignments due if we asked for revisions!
• Seven Days of Git: Day 7 of 7: Record completion on Canvas as first test.

Beginning from today through Wednesday 2/17:

• Post proposal ideas (each in its own distinct post) in the DIGIT-Coders Slack channel #digit210-projectideas for team projects to work on this semester.
• Each student should post an idea for the class to consider: a project involving large text or collection of texts to be managed within a team of 2-4 students to investigate something we could study and visualize from our markup more effectively with computers than with human reading and description alone.
• Proposals should identify and link to a source of texts you want to work with available on the public web.
• All projects must involve a team of at least two persons, but this first exploratory proposal is an individual assignment.
• Each student must respond to at least one of the proposed ideas from another student and indicate suggestions or further ideas.
• You may respond to more than one if you like, and indicate which proposals interest you to work on.
• Proposal discussions will run until class time on F 2/19 when we will form project teams.

Read Intro to Relax NG

• Post in Project ideas and options on DIGIT-Coders Slack

• Relax NG Exercise 1 for classtime
• Complete Take-home Regex Test (by end of the day).
• Read (and indicate on Canvas that you have read) our Requirements for initiating semester projects (by end of the day).

• Post project your project idea(s) by today in the DIGIT-Coders Slack (#digit210-projectideas channel)

• Relax NG exercise 2

Project Checkpoint 1: Launch the project GitHub repo and invite your teammates and me to join (using Settings > Manage settings). Launch Slack channel for project and invite teammates and Dr. B. Post in your Slack project thread your available meeting times to help determine a regular meeting time for your group.

HTML/CSS Exercise 2

• Read about HTML Accessibility and apply what you learn about accessible code on your HTML code for headings, images (providing alt attributes), links, declaring the language. Try applying title attributes.
• Read about Responsive HTML and try applying what you learn to scaling some elements on your site.

The files go on your webspace: Provide the published web link to your files on Canvas.

• Access newtfire eXist-dB
• Download and install eXist-dB on your own computer

• Work with our Follow the XPath tutorial and work through XPath Exercise 1
• Project teams work toward Project Checkpoint 2.

• Read and check out some things you can try in our newtFire eXist environment at Michael Kay’s Learn XQuery in 10 Minutes tutorial and our Introduction to XQuery and the eXist XML Database.
• (Due before class) XQuery Exercise 1: XPath over a collection
• Project Checkpoint 2 (due by the end of the day):
  • Create a file directory structure for the project GitHub repo(s): Initiate the project website within the docs directory with an index.html page and some CSS. Consult with your team and Dr. B to decide on a place to work on the text files (in its own directory, or in a separate private repo?) and create that space. Create a directory for XML files. Begin populating those file directories (even with placeholder Readme.md files to describe what belongs where).
  • Assemble the text files you want to work with on the project. As a team, work on document analysis to plan for how you want these to be marked for structure. What XML structure do you want to use to contain meaningful units of text data? Aim for a clear, simple structure that distinguishes the kind of info you want to be able to track.

• Follow our instructions to install eXist-dB locally on your computer. If you run into trouble, shout out on Slack! Upload the Disney Songs collection (from our textAnalysis-Hub to work with in your local eXist-dB, and try starting your XQuery for Exercise 4 here. (If it doesn’t work, just do your XQuery on our newtfire eXist-dB as usual.)
• Work on your project! Check in with your team. Work on preparing file(s) for your project collection to explore with XQuery in preparation for the next milestone (next Wed. 3/17).

• XQuery Exercise 4: Disney Songs data to HTML

• Projects: Midterm Checkpoint
  • All or most of your texts are prepared in XML and (nearly) ready for XQuery and analysis. Or the work remaining to prepare your texts is easily defined on your GitHub Issues for the project.
  • Relax NG schema is prepared and associated with your files. The team is has been error-correcting and proofing the text base.
  • The website is progressing: there is a site menu and more than one page. Some content appears to announce what this project is about and what questions the team is exploring.
  • Some XQuery over all of some of the team XML is present in the team GitHub repo, and some results of that XQuery are shared, if not on the website, at least in the repo.

• Work through the w3schools SVG tutorial to complete the SVG basic and SVG shapes pages. Filters, gradients, examples, and reference are optional.
• SVG Exercise 1

• SVG Exercise 3: Catch up on previous XQuery exercises if you need to repair them. Continue with plotting SVG from Assassin's Creed, this time converting your plot to a bar graph: Prepare evenly-spaced side-by-side bars, so the count of actions is next to the count of distinct speakers. Prepare X and Y axes for your graph, and label your counts.
• Prepare for Network Analysis: Install Cytoscape on your computer. Begin familiarizing yourself with the Cytoscape interface, working with Cytoscape session files (with .cys extension) in the textAnalysis-Hub in Class Examples >> XQuery-NetworkAnalysis. Try opening one of the Cytoscape session files (.cys) found in one of the project directories there following our tutorial instructions.
• Read An Introduction to Network Analysis and Cytoscape for XML Coders

• XQuery to Network Analysis: Exercise 2
• Get ready for the Python unit: Install Jetbrains Pycharm Edu on your computer

• Make sure Jetbrains Pycharm Edu on your computer
• Pycharm Edu: Intro to Python tutorials: starting working through Introduction through Variables

• Project Visualization Milestone:
  • Each team prepares visualizations for the project with SVG and/or Network plots. Store these in the project GitHub repo and the newtfire eXist-dB database and post progress and links to these materials on Canvas
  • Progress on Project Websites:
    • Work on incorporating places for analysis and visualizations on the site.
    • Draft background information about your resources and their origins.
    • Write up and post the research questions the team is exploring.

• Complete Pycharm Edu Community tutorials: Introduction through Strings and submit evidence of completion (via screen capture) on Canvas.
• XQuery Test due by Tuesday 4/6, 11:59pm

• Patrick Juola, The Rowling Case: A Proposed Standard Analytic Protocol for Authorship Questions, Digital Scholarship in the Humanities 30:1 (2015)
• José Nilo G. Binongo, Who Wrote the 15th Book of Oz? An Application of Multivariate Analysis to Authorship Attribution, Chance 16:2 (2003).

• (Within project teams): Prepare text files from your project for the Stylometry workshop: pull a collection of text files, or plan to work wtih files you’ve prepared for Python NLP.
• Work toward the Project Visualization and Documentation Development Milestone