diff --git a/brainsteam/content/annotations/2023/01/29/1674994106.md b/brainsteam/content/annotations/2023/01/29/1674994106.md new file mode 100644 index 0000000..e7d1293 --- /dev/null +++ b/brainsteam/content/annotations/2023/01/29/1674994106.md @@ -0,0 +1,68 @@ +--- +date: '2023-01-29T12:08:26' +hypothesis-meta: + created: '2023-01-29T12:08:26.920806+00:00' + document: + title: + - 2301.11305.pdf + flagged: false + group: __world__ + hidden: false + id: ovUwTp_NEe2lC8uCWsE7eg + links: + html: https://hypothes.is/a/ovUwTp_NEe2lC8uCWsE7eg + incontext: https://hyp.is/ovUwTp_NEe2lC8uCWsE7eg/arxiv.org/pdf/2301.11305.pdf + json: https://hypothes.is/api/annotations/ovUwTp_NEe2lC8uCWsE7eg + permissions: + admin: + - acct:ravenscroftj@hypothes.is + delete: + - acct:ravenscroftj@hypothes.is + read: + - group:__world__ + update: + - acct:ravenscroftj@hypothes.is + tags: + - chatgpt + - detecting gpt + target: + - selector: + - end: 16098 + start: 15348 + type: TextPositionSelector + - exact: "Figure 3. The average drop in log probability (perturbation discrep-ancy)\ + \ after rephrasing a passage is consistently higher for model-generated passages\ + \ than for human-written passages. Each plotshows the distribution of the\ + \ perturbation discrepancy d (x, p\u03B8 , q)for human-written news articles\ + \ and machine-generated arti-cles; of equal word length from models GPT-2\ + \ (1.5B), GPT-Neo-2.7B (Black et al., 2021), GPT-J (6B; Wang & Komatsuzaki\ + \ (2021))and GPT-NeoX (20B; Black et al. (2022)). Human-written arti-cles\ + \ are a sample of 500 XSum articles; machine-generated textis generated by\ + \ prompting each model with the first 30 tokens ofeach XSum article, sampling\ + \ from the raw conditional distribution.Discrepancies are estimated with 100\ + \ T5-3B samples." + prefix: ancy)0.00.20.40.60.81.0Frequency + suffix: to machine-generated text detect + type: TextQuoteSelector + source: https://arxiv.org/pdf/2301.11305.pdf + text: quite striking here is the fact that more powerful/larger models are more + capable of generating unusual or "human-like" responses - looking at the overlap + in log likelihoods + updated: '2023-01-29T12:08:26.920806+00:00' + uri: https://arxiv.org/pdf/2301.11305.pdf + user: acct:ravenscroftj@hypothes.is + user_info: + display_name: James Ravenscroft +in-reply-to: https://arxiv.org/pdf/2301.11305.pdf +tags: +- chatgpt +- detecting gpt +- hypothesis +type: annotation +url: /annotations/2023/01/29/1674994106 + +--- + + + +
Figure 3. The average drop in log probability (perturbation discrep-ancy) after rephrasing a passage is consistently higher for model-generated passages than for human-written passages. Each plotshows the distribution of the perturbation discrepancy d (x, pθ , q)for human-written news articles and machine-generated arti-cles; of equal word length from models GPT-2 (1.5B), GPT-Neo-2.7B (Black et al., 2021), GPT-J (6B; Wang & Komatsuzaki (2021))and GPT-NeoX (20B; Black et al. (2022)). Human-written arti-cles are a sample of 500 XSum articles; machine-generated textis generated by prompting each model with the first 30 tokens ofeach XSum article, sampling from the raw conditional distribution.Discrepancies are estimated with 100 T5-3B samples.
quite striking here is the fact that more powerful/larger models are more capable of generating unusual or "human-like" responses - looking at the overlap in log likelihoods \ No newline at end of file