• kbal@fedia.io
    link
    fedilink
    arrow-up
    0
    ·
    2 months ago

    Made strange choices about what to highlight.

    They certainly do. For a while it was common to see AI-generated summaries under links to articles on lemmy, so I got a feel for them. Seems to me you would not need any fancy artificial intelligence to do equally well: Just take random excerpts, or maybe just read every third sentence.

  • khalid_salad@awful.systems
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Could it be because a statistical relation isn’t the same as a semantic one? No, I must be prompting it wrong. I’ll just add “engineer” to my title and then everyone will take me seriously.

  • z00s@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    2 months ago

    The problem is not the LLMs, but what people are trying to do with them.

    They are currently spoons, but people are desperately wishing they were katanas.

    They work really well for soup, but they can’t cut steak. But they’re being hyped as super ninja steak knives, and people are getting pissed when they can’t cut steak.

    If you give them watery, soupy tasks they can do successfully, they can lighten your workload, as long as you’re aware of what they are and aren’t good at.

    What people want LLMs to be able to do, ie. “Steak” tasks:

    • write complex documents

    • apply complex knowledge/rules to a situation

    • Write complex code and create entire programs based on vague description

    What LLMs can currently do ie. “Soup” tasks:

    • check this document and fix all spelling, punctuation and grammatical errors

    • summarise this paragraph as dot points

    • write a python program that sorts my photographs into folders based on the year they were taken

    Half of Lemmy is hyping katanas, the other half is yelling “Why won’t my spoon cut this steak?!! AI is so dumb!!!”

    • V0ldek@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      What LLMs can currently do summarise this paragraph as dot points

      The entire point here is that they can’t?

      • fuzzzerd@programming.dev
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Clearly this post is about LLMs not succeeding at this task, but anecdotally I’ve seen it work OK and also fail. Just like humans, which is the benchmark but they are faster.

        • self@awful.systems
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          humans are clearly faster at generating utterly banal shit, as proven by your posts in this thread

    • istewart@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      Why did this immediately give me a flashback to Donald Trump yelling, “when it comes to great steaks, I’ve just raised the stakes!

    • FredFig@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      2 months ago

      Food analogy

      This level of discourse wouldn’t fly on 4chan, how is it so popular with LLM fans?

      • David Gerard@awful.systemsOPM
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        needs to be a car analogy

        • What people want LLMs to do, i.e. Corvette tasks
        • What LLMs actually do, i.e. Trabant tasks
        • self@awful.systems
          link
          fedilink
          English
          arrow-up
          0
          ·
          2 months ago

          What LLMs actually do, i.e. Trabant tasks

          more of a Power Wheels Barbie Jeep whose battery got left out in the sun too long, but I’ll allow it

    • froztbyte@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      good god this entire post is the most tortured believer whataboutism I’ve encountered this month and there’s extremely strong competition here

      are currently spoons, but people are desperately wishing they were katanas

      ie. “Steak” tasks

      you should make a youtube channel, The Katana Steak-Eater. I’d watch the shit out of that at least one saturday afternoon

    • self@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      they don’t do any of that soup shit reliably either and reading the article might have told you that

    • blakestacey@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      I’d offer congratulations on obfuscating a bad claim with a poor analogy, but you didn’t even do that very well.

  • beefbot@lemmy.blahaj.zone
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Is it only me, or is the linked article not super long on details & is reaching a conclusion from 2 examples? This is important & I need to hear more, & I’m generally biased against AI at this point— but the article isn’t doing enough to convince me

    • self@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      did you click through to any of the inline citations? David’s shorter articles on pivot mostly gather and summarize those, so if you need to read the original research and its conclusions that’s where to go

  • Scary le Poo@beehaw.org
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    I keep having to remind people. Chatgpt is only as good as the prompt you give it. I am astounded as the amount of garbage that some people get, but I also know that it’s generally because their prompts are garbage.

    Sometimes it’s output sucks, even with good input. But likely, if the output is bad, the input was bad.

  • swlabr@awful.systems
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    ATTN: If you’re coming into this thread to say, “The output of AI is bad because your prompts suck,” I’m just proud that you managed to figure out how to use the internet at all. Good job, you!

    • froztbyte@awful.systems
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      remember remember, eternal september

      (not that I much agree with the classist overtones of the original, but fuck me does it come to mind often)

  • hex@programming.dev
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Facts are not a data type for LLMs

    I kind of like this because it highlights the way LLMs operate kind of blind and drunk, they’re just really good at predicting the next word.

    • CleoTheWizard@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      They’re not good at predicting the next word, they’re good at predicting the next common word while excluding most unique choices.

      What results is essentially if you made a Venn diagram of human language and only ever used the center of it.

      • hex@programming.dev
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Yes, thanks for clarifying what I meant! AI will never create anything unique unless prompted uniquely and even then it will tend to revert back to what you expect most.

  • David Gerard@awful.systemsOPM
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    i have seen the light from the helpful posters here, made up bullshit alleged summaries of documents are great actually

  • lightnsfw@reddthat.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Ok? I don’t have another human available to skim a shitload of documents for me to find answers I need and I don’t have time to do ot myself. AI is my best option.

    • s3p5r@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      2 months ago

      So long as you don’t care about whether they’re the right or relevant answers, you do you, I guess. Did you use AI to read the linked post too?

      • jaemo@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        Yep. Go ahead and ignore all the cases where it’s getting answers correct and actually helping. We’re all just hallucinating, it’s in no way my lived experience. Your reality is the prime reality and we’re the NPC’s.

      • lightnsfw@reddthat.com
        link
        fedilink
        English
        arrow-up
        0
        ·
        2 months ago

        I didn’t read the post at all because its premise is irrelevant to my situation. If I had another human to read documentation for me I would do that. I don’t so the next best thing is AI. I have to double check its findings but it gets me 95% of the way there and saves hours of work. It’s a useful tool.

  • Sibbo@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Well, to be fair, AI can do it in seconds. Which beats humans.

    But if that is relevant if the results are worthless is another question.

  • RagnarokOnline@programming.dev
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    I had GPT 3.5 break down 6x 45-minute verbatim interviews into bulleted summaries and it did great. I even asked it to anonymize people’s names and it did that too. I did re-read the summaries to make sure no duplicate info or hallucinations existed and it only needed a couple of corrections.

    Beats manually summarizing that info myself.

    Maybe their prompt sucks?

  • GBU_28@lemm.ee
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Dang everyone here needs to look at a tree or a cat or something. Energy is wack in here