If someone demands a piece of software that completely erases all available data when a certain button is pressed, no one is told about it, someone presses it by accident and deletes a whole bunch of data, that’s not a software development error. That’s a management failure. The software did exactly what it was supposed to do.
And LLMs wouldn’t make anything better in that regard. An LLM is built to do as the user wishes, with much, much less pushback than a human. So whereas a human getting the demand to write that software might say, “what in the actual fuck are you smoking and can I have some of it”, an LLM will typically say, “that’s a very bold decision!” In fact, in the case of MCAS, the issue of safety was raised by the engineers many times, and was flat out ignored by management. An LLM is much less likely to make a fuss and much more likely to just do it.
Very true. LLMs are not likely to perform any more admirably in the Milgram experiments than people do. Although, with a bit of appropriate pre-prompting, you might regulate LLM output to better comply with regulations. I think people are even more easily intimidated to “do what they are told” both directly and implied.
If you ask LLMs about cybersecurity implementation decisions, and keep the prompts straightforward like: “is this adequately secure for the target environments?” they are coming out on the highly conservative / secure side of things lately. Of course you can prompt and reprompt them with ever more detailed expectations until they agree with whatever you say, but out of the gate a clean context will generally bring you every “best practice” and then some and point out a big list of things your legacy system isn’t doing.
If someone demands a piece of software that completely erases all available data when a certain button is pressed, no one is told about it, someone presses it by accident and deletes a whole bunch of data, that’s not a software development error. That’s a management failure. The software did exactly what it was supposed to do.
And LLMs wouldn’t make anything better in that regard. An LLM is built to do as the user wishes, with much, much less pushback than a human. So whereas a human getting the demand to write that software might say, “what in the actual fuck are you smoking and can I have some of it”, an LLM will typically say, “that’s a very bold decision!” In fact, in the case of MCAS, the issue of safety was raised by the engineers many times, and was flat out ignored by management. An LLM is much less likely to make a fuss and much more likely to just do it.
Very true. LLMs are not likely to perform any more admirably in the Milgram experiments than people do. Although, with a bit of appropriate pre-prompting, you might regulate LLM output to better comply with regulations. I think people are even more easily intimidated to “do what they are told” both directly and implied.
If you ask LLMs about cybersecurity implementation decisions, and keep the prompts straightforward like: “is this adequately secure for the target environments?” they are coming out on the highly conservative / secure side of things lately. Of course you can prompt and reprompt them with ever more detailed expectations until they agree with whatever you say, but out of the gate a clean context will generally bring you every “best practice” and then some and point out a big list of things your legacy system isn’t doing.