Digital Humanities: Text-Mining

Text mining is the process to discover knowledge (or information, patterns) from text data, which are unstructured or semi-structured, by kinds of transformation of a text.
Researchers use text mining tasks such as:

• Textual analytics
• Entity extraction
• Topic modelling
• Natural Language processing

Basic text analysis 

• Word frequency Lists of words and their frequencies.
• Collocation Words commonly appearing near each other
• Concordance The instances and contexts of a given word or set of words.
• N-grams Common two-, three-, etc.- word phrases
• Parts of speech tagging Identifying words as nouns, verbs, adjectives, adverbs, etc. based on both its definition and its context.
• Named entity recognition Identifying names, places, time periods, etc.

Advanced text analysis (From Underwood, T. (2012). Where to start with text mining.)

• Document classification
  • Information retrieval (e.g., search engines)
  • Supervised classification (e.g., guessing genres)
  • Unsupervised clustering (e.g., alternative “genres”)
• Corpora comparison e.g., political speeches
• Language use over time e.g., Google ngram viewer
• Detecting clusters e.g. of document features (i.e., topic modeling)
• Entity recognition/extraction e.g., geoparsing
• Data Visualization

• "Where to start with text mining." Underwood
• Programming Historian
• Topic Modelling: A Basic Introduction

Datasets are the basis for getting started a text mining project. Some textual data application programming interfaces (APIs) / platforms are openly accessible to the public and some of them are available to the HKU community.

Government and institution data

• Data.gov
• data.gov.hk
• data.gov.uk
• OECD Data
• Open Canada
• opendata.swiss
• The World Bank
• UNICEF DATA
• World Health Data Plarform

Open data for research

• Caselaw Access Project
• Digital Public Library of America
• Europeana
• Library of Congress
• Metropolitan Museum of Art Collection
• IEEE Xplore
• OCLC WorldCat Search
• Google Public Data
• WorldData.AI

Other Subscription-based API

• HathiTrust Data
• JSTOR Data for Research (DfR)