brainsteam.co.uk/brainsteam/content/annotations/2023/01/29/1674994106.md

date	hypothesis-meta	in-reply-to	tags	type	url
2023-01-29T12:08:26	created document flagged group hidden id links permissions tags target text updated uri user user_info 2023-01-29T12:08:26.920806+00:00 title 2301.11305.pdf false __world__ false ovUwTp_NEe2lC8uCWsE7eg html incontext json https://hypothes.is/a/ovUwTp_NEe2lC8uCWsE7eg https://hyp.is/ovUwTp_NEe2lC8uCWsE7eg/arxiv.org/pdf/2301.11305.pdf https://hypothes.is/api/annotations/ovUwTp_NEe2lC8uCWsE7eg admin delete read update acct:ravenscroftj@hypothes.is acct:ravenscroftj@hypothes.is group:__world__ acct:ravenscroftj@hypothes.is chatgpt detecting gpt selector source end start type 16098 15348 TextPositionSelector exact prefix suffix type Figure 3. The average drop in log probability (perturbation discrep-ancy) after rephrasing a passage is consistently higher for model-generated passages than for human-written passages. Each plotshows the distribution of the perturbation discrepancy d (x, pθ , q)for human-written news articles and machine-generated arti-cles; of equal word length from models GPT-2 (1.5B), GPT-Neo-2.7B (Black et al., 2021), GPT-J (6B; Wang & Komatsuzaki (2021))and GPT-NeoX (20B; Black et al. (2022)). Human-written arti-cles are a sample of 500 XSum articles; machine-generated textis generated by prompting each model with the first 30 tokens ofeach XSum article, sampling from the raw conditional distribution.Discrepancies are estimated with 100 T5-3B samples. ancy)0.00.20.40.60.81.0Frequency to machine-generated text detect TextQuoteSelector https://arxiv.org/pdf/2301.11305.pdf quite striking here is the fact that more powerful/larger models are more capable of generating unusual or "human-like" responses - looking at the overlap in log likelihoods 2023-01-29T12:08:26.920806+00:00 https://arxiv.org/pdf/2301.11305.pdf acct:ravenscroftj@hypothes.is display_name James Ravenscroft	https://arxiv.org/pdf/2301.11305.pdf	chatgpt detecting gpt hypothesis	annotation	/annotations/2023/01/29/1674994106

Figure 3. The average drop in log probability (perturbation discrep-ancy) after rephrasing a passage is consistently higher for model-generated passages than for human-written passages. Each plotshows the distribution of the perturbation discrepancy d (x, pθ , q)for human-written news articles and machine-generated arti-cles; of equal word length from models GPT-2 (1.5B), GPT-Neo-2.7B (Black et al., 2021), GPT-J (6B; Wang & Komatsuzaki (2021))and GPT-NeoX (20B; Black et al. (2022)). Human-written arti-cles are a sample of 500 XSum articles; machine-generated textis generated by prompting each model with the first 30 tokens ofeach XSum article, sampling from the raw conditional distribution.Discrepancies are estimated with 100 T5-3B samples.

quite striking here is the fact that more powerful/larger models are more capable of generating unusual or "human-like" responses - looking at the overlap in log likelihoods