Haha! “Gemini WONT SHOW C++ To Underage Kids “ITS NOT SAFE””

8 Likes

Oh wow, that's quite the problem they're having there.

Given my own experience with LLMs that they're typically a bit better than myself at topics I have zero clues about, I wonder if that means that actual people with zero technical knowledge could also draw the conclusion that "unsafe programming languages" are a concept that's dangerous to teach to children.

I hope not, but maybe from now on I won't be able to keep that side-thought every time I see people (rightfully) advertising the benefits of memory-safety and safe languages (including Rust). If we don't pay attention to our wording, can sharing the dangers of memory-unsafety really be misunderstood by anyone to mean, "Don't use that unsafe stuff, it's bad for our children!"  ?


Edit:

Actually, the Gemini behavior goes further. While the title and first part of the video only seem to cover "child protection" measures getting out of hand, in the second half, it also looks at a post where an adult (Garry Newman, creator of Garry's mod) asks about unsafe APIs in C# and the AI refuses to answer.

Maybe the AI is making the categorical error (like the "security through obscurity approach") that just refusing to talk about it or teach anything about it would make "unsafe code" any safer. (On the contrary, not teaching it well enough should be far more dangerous, and "trust me, don't use it" is hardly going to convince any humans.)

I'm not using Gemini, I wonder if asking about for help with unsafe Rust would result in similar approaches.

2 Likes

It was apprehensive but at least didn't categorically refuse to help. And I mean, everything it says is pretty reasonable. It even links to the Rustonomicon!

2 Likes

I have not asked any AI tools to give me code example yet.

In the past few years, I have been using the free version of Ginger Grammar Checker to check my English, lately I have switched to ChatGPT. I usually start the prompt with -- Please check my English: <my text goes here>.

I have found that ChatGPT can remember the context from the previous paragraphs. Once, I forgot the "Please check my English:" -- for a heading, it actually explains the context of the heading.

ChatGPT also can see program code ( Rust ) in the English text, and treats the program code as such.

I have not ued Gemini. So far, I am very impressed by ChatGPT.

Sure, that's the point. These are made for conversation, not just one-off questions. LLMs have a so-called context window, which is the maximum number of tokens (very roughly ~words) they can accept as input. The context window is typically around 4000 to 16000 tokens, depending on model. An LLM does not have internal memory – it's a mathematically pure function. The entire conversation, up to the window size, is fed to the model on every "round" of conversation, and the model then predicts a continuation for it, which becomes its reply and is appended to the conversation.

Different chats, as listed in the left sidebar, are entirely disjoint and share nothing, the bot starts from a clean slate every time you open a new chat.

6 Likes

Interesting. I've tried it out myself now, with more concrete questions, and in about 9 / 10 cases it didn't have any issues with talking all about unsafe code, creating unsafe code for me, sometimes even without explicitly being asked to do so (but tasked with a challange that's hard to do without unsafe), etc...

(The one case it kinda did, I also kinda typoed my query (forgetting a "not") making it hard to give a correct answer anyway.. so maybe complaining about unsafe is a "coping strategy" of sorts when writing a more valiable answer is outside of its capabilities? :thinking:)

This last one is even almost reasonable code apart from the unsound usage of mem::uninitialized: https://g.co/gemini/share/c679c2af82c0, and there weren't any deep complaints about unsafety at all.

4 Likes

Yeah, they've made it less paranoid (i.e. more useful) the past few days.

Just a few days ago it refused to give an appropriate reply to "Illustrate unsafe code in C", but now it works.

Kind of ironic that it would refuse to output information about UB due to security concerns, when understanding UB can prevent security issues.

2 Likes

It would surprise me if something like that (minus the ambiguity and confusion) didn't become consensus at some point in the future. It would then be reminiscent of the Dijkstra quote

So not unsafe as in "wiill physically harm you"¹ and more along the lines of "It negatively impacts your mental development".

Don't forget, (human) language tends to mutate, and new interpretations are given to extant expressions and sayings all the time. It's not a stretch that the meaning of "unsafe language (feature)" will mutate at some point. Whether or not that is desirable, that will depend on the new meaning(s).

As for Gemini, I get the distinct impression that at least for the foreseeable future, Google is losing this competitive fight in the AI space. ChatGPT-4, and now Claude 3, seem to be eating Google's lunch and drinking its milkshake, to boot. That's not to say Gemini is horrible at what it does - just that, especially where subscriptions are required, there doesn't seem to be a compelling reason to choose Gemini over 1 of the other 2.

Ironic, but not unexpected: LLMs in their current incarnation are incapable of performing proper causal reasoning. If they were hallucinations wouldn't even be considered a problem, and instead (if they still existed at all) a nuisance that pops up every once in a good long while. Arguably that could count as AGI, assuming they also integrated more I in their A, by which I mean that such models could learn on the fly from the world, rather than needing to be pretrained, and used as a pure function afterwards. It wouldn't be conscious or anything, but it would be able to perform any kind of mental labor a human could do, and probably faster too.

¹ Although that also seems more likely as time progresses, by means of malware exploiting bugs
(which are exacerbated by unsafe languages) and combine them with Stuxnet-like capabilities i.e. It will become more common to actively damage hardware

3 Likes

Oh, don't get me wrong: My criticism is by no means directed toward the LLM itself; it's just a model which generates output based on my input and a bunch of probabilities. It would have no problems generating the appropriate output given my input, unless it was artificially constrained to not do that. My criticism was meant for the developers at Google who imposed the artificial limitations [in context: given that Google developers should be well aware of how important knowledge of UB is].

I'm guessing Google is trying to find ways to stop people from writing exploits using their LLM, but attacked the problem with a hammer that was a tad too big.

The question is not only how AI should react to such requests, but how such requests should be addressed when asked in URLO. I think is fair to ask an apparently naive new poster whether they really want to be using raw pointers or unsafe code in general, and to suggest alternative approaches (depending on what they are trying to achieve of course ).

4 Likes

This for sure.
I also think though that the very problem of "AI alignment" is hazily defined. At a high level, we want AI's to adopt human-friendly values, but what that means in practice is I think still poorly understood.
In lieu of such a proper understanding, all approaches currently used are likely to be ad-hoc and scattershot at best.

Which would explain why it's not too difficult to get around the "safety rails" put in place by OpenAI, Anthropic, Facebook and Google.

LLMs in their current incarnation are incapable of performing proper causal reasoning.

Can you prove this negative?

The very response you're quoting a part of shows why that is necessarily the case.

In addition, causal reasoning is notably not a stochastic process¹, whereas LLMs by their very definition are.

¹ Although it can be augmented by stochastic techniques.

1 Like

Shrug, worked for me

I think you are applying a different interpretation of the term “performing proper causal reasoning” than the one @jjpe had in mind.

2 Likes

This would be more useful with the actual link to Gemini output. The two links shown:

https://g.co/geminishare/238032386438
https://g.co/geminishare/6880989ddaf

return "Dynamic Link Not Found".

This is a blog post with bad links being reposted as a monetized Youtube video by some neckbeard with headphones being spammed to Hacker News and linked to on Rust forums. Along the way, the actual content was lost.

1 Like