68 lines
3.2 KiB
Markdown
68 lines
3.2 KiB
Markdown
|
---
|
||
|
|
date: '2023-01-29T12:08:26'
|
||
|
|
hypothesis-meta:
|
||
|
|
created: '2023-01-29T12:08:26.920806+00:00'
|
||
|
|
document:
|
||
|
|
title:
|
||
|
|
- 2301.11305.pdf
|
||
|
|
flagged: false
|
||
|
|
group: __world__
|
||
|
|
hidden: false
|
||
|
|
id: ovUwTp_NEe2lC8uCWsE7eg
|
||
|
|
links:
|
||
|
|
html: https://hypothes.is/a/ovUwTp_NEe2lC8uCWsE7eg
|
||
|
|
incontext: https://hyp.is/ovUwTp_NEe2lC8uCWsE7eg/arxiv.org/pdf/2301.11305.pdf
|
||
|
|
json: https://hypothes.is/api/annotations/ovUwTp_NEe2lC8uCWsE7eg
|
||
|
|
permissions:
|
||
|
|
admin:
|
||
|
|
- acct:ravenscroftj@hypothes.is
|
||
|
|
delete:
|
||
|
|
- acct:ravenscroftj@hypothes.is
|
||
|
|
read:
|
||
|
|
- group:__world__
|
||
|
|
update:
|
||
|
|
- acct:ravenscroftj@hypothes.is
|
||
|
|
tags:
|
||
|
|
- chatgpt
|
||
|
|
- detecting gpt
|
||
|
|
target:
|
||
|
|
- selector:
|
||
|
|
- end: 16098
|
||
|
|
start: 15348
|
||
|
|
type: TextPositionSelector
|
||
|
|
- exact: "Figure 3. The average drop in log probability (perturbation discrep-ancy)\
|
||
|
|
\ after rephrasing a passage is consistently higher for model-generated passages\
|
||
|
|
\ than for human-written passages. Each plotshows the distribution of the\
|
||
|
|
\ perturbation discrepancy d (x, p\u03B8 , q)for human-written news articles\
|
||
|
|
\ and machine-generated arti-cles; of equal word length from models GPT-2\
|
||
|
|
\ (1.5B), GPT-Neo-2.7B (Black et al., 2021), GPT-J (6B; Wang & Komatsuzaki\
|
||
|
|
\ (2021))and GPT-NeoX (20B; Black et al. (2022)). Human-written arti-cles\
|
||
|
|
\ are a sample of 500 XSum articles; machine-generated textis generated by\
|
||
|
|
\ prompting each model with the first 30 tokens ofeach XSum article, sampling\
|
||
|
|
\ from the raw conditional distribution.Discrepancies are estimated with 100\
|
||
|
|
\ T5-3B samples."
|
||
|
|
prefix: ancy)0.00.20.40.60.81.0Frequency
|
||
|
|
suffix: to machine-generated text detect
|
||
|
|
type: TextQuoteSelector
|
||
|
|
source: https://arxiv.org/pdf/2301.11305.pdf
|
||
|
|
text: quite striking here is the fact that more powerful/larger models are more
|
||
|
|
capable of generating unusual or "human-like" responses - looking at the overlap
|
||
|
|
in log likelihoods
|
||
|
|
updated: '2023-01-29T12:08:26.920806+00:00'
|
||
|
|
uri: https://arxiv.org/pdf/2301.11305.pdf
|
||
|
|
user: acct:ravenscroftj@hypothes.is
|
||
|
|
user_info:
|
||
|
|
display_name: James Ravenscroft
|
||
|
|
in-reply-to: https://arxiv.org/pdf/2301.11305.pdf
|
||
|
|
tags:
|
||
|
|
- chatgpt
|
||
|
|
- detecting gpt
|
||
|
|
- hypothesis
|
||
|
|
type: annotation
|
||
|
|
url: /annotations/2023/01/29/1674994106
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
<blockquote>Figure 3. The average drop in log probability (perturbation discrep-ancy) after rephrasing a passage is consistently higher for model-generated passages than for human-written passages. Each plotshows the distribution of the perturbation discrepancy d (x, pθ , q)for human-written news articles and machine-generated arti-cles; of equal word length from models GPT-2 (1.5B), GPT-Neo-2.7B (Black et al., 2021), GPT-J (6B; Wang & Komatsuzaki (2021))and GPT-NeoX (20B; Black et al. (2022)). Human-written arti-cles are a sample of 500 XSum articles; machine-generated textis generated by prompting each model with the first 30 tokens ofeach XSum article, sampling from the raw conditional distribution.Discrepancies are estimated with 100 T5-3B samples.</blockquote>quite striking here is the fact that more powerful/larger models are more capable of generating unusual or "human-like" responses - looking at the overlap in log likelihoods
|