In T376306 we identified popular document editors when pasting so as not to trigger a copyright paste check on them.
Conversely we may want a stronger warning when content is pasted from popular LLM chatbot websites, if they can be detected.
It is likely that the HTML output of these sites will change more quickly over time compared to document editors, so any detectors would need to be regularly updated.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T265163 Create a system to encode best practices into editing experiences | |||
| In Progress | nayoub | T359107 Paste Check: Prompt people pasting text to consider risk of copyright violation | |||
| Resolved | ppelberg | T376306 Identify paste source from copied HTML | |||
| Resolved | Esanders | T379908 Identify pastes from popular LLMs |
Most sites have a "copy" button as well as the ability to just select the text on the page, so we can document these cases separately:
| LLM | Copy button | Select and copy[1] |
|---|---|---|
| Gemini | Same as select and copy | Appears to have the distinctive attribute data-sourcepos, e.g. <p data-sourcepos="9:1-9:117">.... This may come from some generic library though, so not necessarily a solid indicator |
| ChatGPT | Plain text, unidentifiable | Contains some attributes starting data-message- such as data-message-author-role="assistant", data-message-id and data-message-model-slug="gpt-4o" |
| Claude | Plain text, unidentifiable | Very plain HTML, but specific class list might be fairly unique: <p class="whitespace-pre-wrap break-words"> |
Through this investigation, we've learned that:
The above has led us to conclude that, for now, we do not think the HTML included in pastes from popular LLMs is stable enough for Paste Check to be configured off of it.
Although, if/when the HTML does stabilize we'll revisit this choice and we'll use T382608 to hold us accountable to doing so.
Reviewers are starting to get quite overwhelmed with poor LLM additions. Has this stabilised enough to reconsider? Policies have become more clear as well that new content creation via LLM should not happen. When I tested copying some LLM text as a TA, I got a warning about plagiarism, which isn't that appropriate.
+1 would be great if PasteCheck could display a distinct warning if HTML indicates that the content source is an LLM. Using TextMatch is not sufficient given the high risk of false positives when including too many potential keywords.