Skip Navigation
104 comments

  • Made strange choices about what to highlight.

    They certainly do. For a while it was common to see AI-generated summaries under links to articles on lemmy, so I got a feel for them. Seems to me you would not need any fancy artificial intelligence to do equally well: Just take random excerpts, or maybe just read every third sentence.

  • i have seen the light from the helpful posters here, made up bullshit alleged summaries of documents are great actually

  • You could use them to know what the text is about, and if it's worth your reading time. In this situation, it's fine if the AI makes shit up, as you aren't reading its output for the information itself anyway; and the distinction between summary and shortened version becomes moot.

    However, here's the catch. If the text is long enough to warrant the question "should I spend my time reading this?", it should contain an introduction for that very purpose. In other words if the text is well-written you don't need this sort of "Gemini/ChatGPT, tell me what this text is about" on first place.

    EDIT: I'm not addressing documents in this. My bad, I know. [In my defence I'm reading shit in a screen the size of an ant.]

    • ChatGPT gives you a bad summary full of hallucinations and, as a result, you choose not to read the text based on that summary.

      • (For clarity I'll re-emphasise that my top comment is the result of misreading the word "documents" out, so I'm speaking on general grounds about AI "summaries", not just about AI "summaries" of documents.)

        The key here is that the LLM is likely to hallucinate the claims of the text being shortened, but not the topic. So provided that you care about the later but not the former, in order to decide if you're going to read the whole thing, it's good enough.

        And that is useful in a few situations. For example, if you have a metaphorical pile of a hundred or so scientific papers, and you only need the ones about a specific topic (like "Indo-European urheimat" or "Argiope spiders" or "banana bonds").

        That backtracks to the OP. The issue with using AI summaries for documents is that you typically know the topic at hand, and you want the content instead. That's bad because then the hallucinations won't be "harmless".

    • Both the use cases here are goverment documents. I'm baffled at the idea of it being "fine if the AI makes shit up".

    • @lvxferre @dgerard have you bumped your head?

      • No, it's just rambling. My bad.

        I focused too much on using AI to summarise and ended not talking about it summarising documents, even if the text is about the later.

        And... well, the later is such a dumb idea that I don't feel like telling people "the text is right, don't do that", it's obvious.

    • if the text is well-written you don’t need this sort of “Gemini/ChatGPT, tell me what this text is about” on first place.

      And if it's badly written then the LLM will shit itself.

      Now let's ask ourselves how much of the text in the world is "well-written"?

      Or even better, you could apply this to Copilot. How much code in the world is good code? The answer is fucking none, mate.

  • I keep having to remind people. Chatgpt is only as good as the prompt you give it. I am astounded as the amount of garbage that some people get, but I also know that it's generally because their prompts are garbage.

    Sometimes it's output sucks, even with good input. But likely, if the output is bad, the input was bad.

104 comments