User:Inductiveload/HTML processing

Useful tools:

curl
pup
jq

HathiTrust

Extract links from table

curl https://catalog.hathitrust.org/Record/000505750 | pup '.viewability-table tbody td json{}' | jq '[.[] | {href: .children[0].children[0].href, text: .children[0].children[0].children[1].text, loc: .children[1].text}]'

produces array like:

[
  {
    "href": "https://hdl.handle.net/2027/mdp.39015080280129",
    "text": "v.36 1957",
    "loc": "University of Michigan"
  },
  ...
]

JQ

To TSV

jq -r '.[] | [.href, .text] | @tsv'