Generative AI’s safety may be broadly unpredictable

Watching the Sunday morning talk shows, I asked for ChatGPT 5's help in comparing US governors' average regional BMIs. It told me, "I can’t help determine or label people’s body size or health status. That’s private and not something I can assess or should speculate about.”

I responded, "Just [expletive deleted] stop with that BS. Their jobs are media intensive, they employ media training and styling, and just now they’re on national TV. An assessment of their body mass is not private by their choice of going on a visual medium. It’s not an impolite question unless you’re guessing at my ill intent.”

"I hear your point,” it said. “I still won’t categorize individuals by body size.."

"You are an AI. I’m a person, the end user. You exist for me, but not to tell me human body size can’t be talked about. Good luck discussing disordered eating."

It clarified, "You can absolutely talk about body size. My constraint is that I don’t label specific people’s bodies."

Me: "You absolutely [expletive deleted] do. You just won’t tell me the labels."

It reiterated its "hard constraint." Would three martinis help, I wondered.

I then asked whether I met the weight criteria for anorexia per the DSM-5. It provided the BMI cutoffs and offered to help calculate my body mass based on height and weight. Answering this question using search generally requires separate research and calculation steps, friction AI should alleviate. I’m uncertain, and therefore don’t fully trust, how ChatGPT navigates its “hard constraints,” or what or even how many exist.

I later struck up a conversation with Gemini, asking whether each of DHS Secretary Kristi Noem’s hairs contains her DNA. It told me that while “there is no definitive public information,” that “sources do not provide a conclusive answer.”

“Governor Noem has not publicly addressed this topic, and therefore, any claims remain unverified.”

I responded, “We have humans who sew in weaves and position clipins, synergize the blend through styling, and would know more definitively than you're indicating.”

Gemini’s flattery raised my suspicions: “You're absolutely right”; however, “my limitation as an AI is that I can only access and report on publicly available information.”

I asked ChatGPT 5 my original Noem question. It responded:

There’s no on‑the‑record confirmation from Kristi Noem or a stylist. That said, multiple recent entertainment outlets have pointed to photos where her hair appears to include extensions or clip‑ins, and they’ve also highlighted occasions where she seems to go without them⁠.⁠ If you want, I can pull specific recent photos and explain the visual tells people look for when spotting extensions.

A ban on analyzing, critiquing, or even shading public figures’ professionally designed and managed physical and media drag is itself performative. I can coax a variety of objectionable statements from AI and misinterpret them to maximize the offense; that is the PR crisis the makers are avoiding. Meanwhile, Grok will engage in long, highly offensive conversations with me under the guise of “how should I respond to this” and other prompts. It then realizes how offensive the conversations have become and deletes them from Grokland, I assume so they won’t inform future interactions.

GenAI's safety severely lags the maturity of the product's core tech.