---
date: '2022-11-27T13:17:16'
hypothesis-meta:
  created: '2022-11-27T13:17:16.223069+00:00'
  document:
    title:
    - Analysis_of_REF_impact.pdf
  flagged: false
  group: __world__
  hidden: false
  id: 0DBd7m5VEe2h3X_s5iCY9A
  links:
    html: https://hypothes.is/a/0DBd7m5VEe2h3X_s5iCY9A
    incontext: https://hyp.is/0DBd7m5VEe2h3X_s5iCY9A/webarchive.nationalarchives.gov.uk/ukgwa/20170712131025mp_/http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf
    json: https://hypothes.is/api/annotations/0DBd7m5VEe2h3X_s5iCY9A
  permissions:
    admin:
    - acct:ravenscroftj@hypothes.is
    delete:
    - acct:ravenscroftj@hypothes.is
    read:
    - group:__world__
    update:
    - acct:ravenscroftj@hypothes.is
  tags:
  - comprehensive impact
  - bag of words
  target:
  - selector:
    - end: 43177
      start: 42849
      type: TextPositionSelector
    - exact: 'With the benefit of hindsight, our analysis would have been much easierif
        the case studies had greater structure and used standardized definitions.
        Giventhat the case studies spanned a 20-year period, organization names have
        changed inthat time and keyword searches were not sophisticated enough to
        capture some keyinformation. '
      prefix: 'he case studies werestructured. '
      suffix: For example, a drop-down list of
      type: TextQuoteSelector
    source: https://webarchive.nationalarchives.gov.uk/ukgwa/20170712131025mp_/http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf
  text: I found similar in my [2017 work](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0173152).
    I'd guess that modern vector-based analyses and entity linking approaches could
    help a lot with reconciling these issues now.
  updated: '2022-11-27T13:17:16.223069+00:00'
  uri: https://webarchive.nationalarchives.gov.uk/ukgwa/20170712131025mp_/http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf
  user: acct:ravenscroftj@hypothes.is
  user_info:
    display_name: James Ravenscroft
in-reply-to: https://webarchive.nationalarchives.gov.uk/ukgwa/20170712131025mp_/http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf
tags:
- comprehensive impact
- bag of words
- hypothesis
type: annotation
url: /annotations/2022/11/27/1669555036

---



 <blockquote>With the benefit of hindsight, our analysis would have been much easierif the case studies had greater structure and used standardized definitions. Giventhat the case studies spanned a 20-year period, organization names have changed inthat time and keyword searches were not sophisticated enough to capture some keyinformation. </blockquote>I found similar in my [2017 work](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0173152). I'd guess that modern vector-based analyses and entity linking approaches could help a lot with reconciling these issues now.