• Sgagvefey@lemmynsfw.com
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    4 months ago

    We do fully understand them. Not knowing the exact reason they come to a model doesn’t mean the algorithm has a shred of mystery involved. It’s like saying we don’t understand fluid dynamics because it’s computationally heavy.

    It’s autocomplete with a really big training set and a really big model. It cannot possibly develop agency. It’s hundreds of orders of magnitude of complexity short of a human.

    • 5C5C5C@programming.dev
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      4 months ago

      That’s not what an algorithms researcher means when we talk about “understanding”. Obviously we know the mechanism by which it operates, it’s not an unknown alien technology that dropped into our laps.

      Understanding an algorithm means being able to predict the characteristics of its outputs based on the characteristics of its inputs. E.g. will it give an optimal solution to a problem that we pose? Will its response satisfy certain constraints or fall within certain bounds?

      Figuring this stuff out for foundation models is an active area of research, and the absence of this predictability is an enormous safety concern for any use cases where the output can be consequential.

      It cannot possibly develop agency.

      I don’t believe I’ve suggested anywhere that I think it will, but I’ll play around with this concern anyway… There’s a lot of discussion going on about having models feed back on themselves to learn from their own output. I don’t find it all that hard to imagine that something we could reasonably consider self awareness could be formed by a very complex neural network that is able to consume and process its own outputs. And once self awareness starts to form, it’s not that hard for me to imagine a sense of agency following. I have no idea what the model might use that agency for, but I don’t think it’s all that far fetched to consider the possibility of it happening.

      • Sgagvefey@lemmynsfw.com
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        1
        ·
        4 months ago

        There are plenty of nondeterministic algorithms. It’s not a special trait. There are plenty of algorithms with actual emergent behavior, which LLMs don’t have to any meaningful extent. We absolutely understand how LLMs work

        The answer to both of your questions is not some unsolved mystery. It’s “of course not”. That’s not what they do and fundamentally requires a much more complex architecture to even approach.

        • 5C5C5C@programming.dev
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          1
          ·
          4 months ago

          Non-deterministic algorithms such as Monte Carlo methods or simulated annealing can still be constrained to an acceptable state space. How to do this effectively for LLMs is a very open question, largely because the state space of the problems that they are applied to is incomprehensibly huge.

          • Sgagvefey@lemmynsfw.com
            link
            fedilink
            English
            arrow-up
            2
            arrow-down
            1
            ·
            4 months ago

            It’s only an “open question” if you are somehow confused by the fact that it’s a super simple algorithm that cannot ever possibly be used like that.

            It may be a small part of a proper architecture for a functional solution, but there’s no possibility that it will ever be doing the heavy lifting. It is what it is, and that’s an obvious dead end.

            • 5C5C5C@programming.dev
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              1
              ·
              edit-2
              4 months ago

              Literally nothing you’ve said gives any indication that you actually know the current state of foundation model research. I won’t claim it’s my research specialty, but I work directly with people whose full time job is research and tuning on foundation models, and everything I’m saying is being relayed from conversations that I have with them.

              “Cannot ever possibly be used like that”… Like what specifically? To drive a car? That’s being done. To give financial advice? That’s being done. To console people who are suicidal or at risk of harming themselves? That’s being done. To make kill / no kill decisions in an active warzone? It’s being considered (if not already being done in secret).

              This technology is being used in extremely consequential positions despite having very weak guarantees around safety. This should give any reasonable person pause. I’m not taking any firm stance on whether this specific regulation is the right approach, but if you think there should be no legal accountability for the outcomes of how this technology gets used then I guess you’re someone who thinks seatbelts should be optional in cars and it’s okay for airplanes to fall out of the sky due to neglect.

              • Sgagvefey@lemmynsfw.com
                link
                fedilink
                English
                arrow-up
                2
                arrow-down
                1
                ·
                4 months ago

                For anything where you would ever expect a predictable, useful outcome to an arbitrary input. There is no possible path to LLMs ever doing anything close to that.

                LLMs aren’t driving cars. LLMs aren’t doing financial modeling. Those are entirely different tools with heavily hand crafted models to specific applications.

                Anyone using an LLM to provide therapy should get multiple life sentences in prison regardless of outcomes. There is no possible way to LLMs ever being actually useful for therapy. It’s just a random text generator that’s tuned well enough to sound good. It has no substance and the underlying tech cannot possibly develop substance.

                • 5C5C5C@programming.dev
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  arrow-down
                  1
                  ·
                  edit-2
                  4 months ago

                  I can’t tell if you’re suggesting that foundation models (which is the underpinning technology of LLMs) aren’t being used for the things that I said they’re being used for, but I can assure you they are, either in commercial R&D or in live commercial products.

                  The fact that they shouldn’t be used for these things is something we can certainly agree on, but the fact remains that they are.

                  Sources:

                  So this all goes back to my point that some form of accountability is needed for how these tools get used. I haven’t examined the specific legislation proposal enough to give any firm opinion on it, but I think it’s a good thing that the conversation is happening in a serious way.