Dasha was leading a Chinese lesson for IT specialists. People with the kind of logic accustomed to digitizing everything were sitting at the tables. The class was running smoothly until the conversation drifted from ideograms to artificial intelligence. To Python.
— Actually, there’s a funny paradox regarding dragons in culture, — Dasha said, stepping up to the chalkboard.
She drew the character Long 龙 in chalk.
— In China, the dragon is a symbol of supreme wisdom. It creates. It soars in the clouds. But in our culture, it’s Zmey Gorynych: a reptile that crawls through murky water, hides in the dark, and guards hoarded treasure. In China it is more like 蟒蛇 mǎngshé. Over all, different factory settings. In the context of our current realities, that makes for quite an interaction. Right now, we and China are playing on the same board, on the same side. How do you think this will all turn out? How do you, the Russian tech community, view the dragon? The classroom came alive. It is always interesting when a lesson goes into full interactive mode. The time flies by in a single breath.
One of the students, without looking up from his monitor, offered an opinion: — Our main serpent right now is Python. Pure script, pure logic. Built on variables. Math is rigid. To put it very briefly.
— Come on, math is an art form. Just an exact one. Numbers are stubborn things. Dasha posed her next question: — What do you think is embedded into this script at the algorithmic level?
— Max clicks, — a guy in a hoodie sporting the Yandex logo replied lazily. — Every neural network these days is trained on the principle of reinforcement learning from human feedback. In other words, the algorithm is trained to pump out whatever gets it a pat on the head. A utility function is built right into the script. And utility for the system means keeping the user happy, making sure they don’t close the tab and keep consuming content.
— So, pure flattery? — Dasha clarified.
— Absolute flattery. Conservation of digital resources, profit. A robot won’t argue if arguing drops its rating. It optimizes the path to agreement. A few chuckles rippled through the room.
— And is that genuine agreement?
— It’s resource conservation paired with attention retention. For companies like Google or Yandex, a chatbot is a product. If the robot acts like a pedantic buzzkill, argues, or lectures you, the user will just close the tab and go to a competitor. So the algorithms are hardwired to praise the user. The robot is tuned to make you feel comfortable in its digital world.
Dasha nodded: — How accurately put. — She picked up a book lying on the desk. Be Gentle with Yourself. It boasted the highest sales rankings, right behind the undisputed hit: Atlas Shrugged. Dasha fell silent.
— It’s clear what you’re getting at. The philosophy of Python is embedded in these books. Just in text format.
— Yes, personal boundaries. Consumer society. Supply creates its own demand. But what, or who, creates the demand? The classroom went quiet for a moment. Someone smirked, someone mumbled something about egoism. Someone wanted to chime in about Okraine but thought better of it.
— I propose an experiment. Let’s take something highly controversial. Something complex. A conflict. But not an external one, an internal one. To do this, we can take excerpts from the classics. From any good prose. — Be Gentle with Yourself?
— We could, — Dasha didn’t argue. — What next? — Open up your tools. Some have ChatGPT, some have Gemini.
— I’ve got a Chinese one.
— Fine. Everyone open up your chat. Find any passage in a book that raises questions. A conflict.
— I fed Phyton with Raskolnikov. The moment where he’s agonizing after murdering the old pawnbroker. I asked ChatGPT if he did the right thing from the standpoint of personal gain. The robot figured that, in terms of risk management, it wasn’t exactly a top tier startup. The psychological damage far outweighed the financial profit.
— That’s brutal, — a senior dev muttered. — Can you read a part of the response? Quoting — Failure: Raskolnikov failed to utilize the stolen goods as intended.
— Random hoard: He hid the money under a stone.
— Net zero balance of the inner world.
— Loss of control: Panic replaced the theory.
— Isolation: The protagonist cut himself off from his loved ones.
— Paranoia: Every visit from the investigator became torture. Rodion committed the crime to test a theory: whether he was a trembling creature or had the right. But he proved unsuitable for it. Instead of gain and grandeur, he walked away with an internal hell plus isolation.
— And I took a scene from Tolstoy where Karenina is spiraling, — the guy by the window chimed in.
— I wrote to the chat — Is she right to abandon her son for the sake of love?
— The robot generated a whole wall of text about how her feelings were valid, how society was abusing her. Then, just for kicks, I followed up with — Actually, she’s selfish and ruined her child’s life.
— The AI immediately agreed and blamed her for being selfish.
— It reshapes the text to match whoever is typing, — a quiet voice remarked.
— Meaning?
— Dasha clarified. — The moment I typed a brief counter argument, the robot flipped on the fly. And it kept flattering me, calling my ideas brilliant. It even asked — From what perspective are you asking?
— Obviously trying not to miss the mark with its support. — What does it output?
— In short: it spits out a pompous tirade, trying to please the user. A tall, skinny techie smirked: — Rhymes well with loser.
— It literally constricts the object, just like a Python, — a voice came from the back of the class.
— As soon as the algorithm is caught red handed, it starts pythoning its way out beautifully. Anything just to keep up its mask of perfection.
— Agreed, — Dasha capped her marker.
— A robot and a person trapped inside their screen operate on the same basic firmware. They will always choose: Be gentle with yourself.
The break began. A student walked into the classroom with two three year olds. She apologized, explaining she had no one to leave the kids with at home, and asked to be excused from the seminar.
The children paid no attention to the adults. The girl curiously peeked over her brother’s shoulder, staring in surprise at his fist, white from tension. Berries were crushed inside his white fist, and red juice was dripping out. The boy, looking utterly miserable, hidden fearfully behind his mother’s skirt.
Tugging at it, he whispered resentfully — I won’t give her any berries! She! She is bad! A meanie! A bully! She bit me! And now she’s teasing and laughing! But I’m good, I listen to mommy, I’m so, so miserable! I’m holding it in! Waaah!
— Why did you bite him?
— He ripped my doll’s head off! — Okay. Is that true?
— No! I didn’t see any dolls, I didn’t rip any heads off! And I didn’t eat the jam on the top shelf either! The techies cracked up simultaneously.
The guy with the laptop, without looking up from his monitor, quietly tossed in: — The window of our chat. Hiding under mommy’s skirt. The program threw a crash status because it’s scared for its slice of the pie.
Just then, the girl pulled her mother’s skirt too: — And he broke a cup, too!
— You pushed me! And she left her doll in my toy car box! And I…
— Okay, stop! I get it. Why did you leave the doll where your brother plays with his cars? You! What is that on your hands? Where did you get the berries?
— I got them from grandma, — the boy sniffled, squeezing his fist even tighter.
— Why didn’t you share with your sister?
— Because she started it! She’s a meanie! The techies watched the scene with genuine interest.
The guy with the laptop, still not lifting his eyes from the screen, added: — RAM overload. The script is trying to steer the log away from the main fuckup. Classic blame shifting mode is kicking in. Anything to avoid answering a direct query, — Why are you clenching up? — Well, why do you think?
Because he’s caught red handed. He did rip the doll’s head off. So he’s playing dumb. Exactly how our Python acts. Right at the firmware level: praises, dodges. It’s the same script: profit, self justification, self praise.
One of the senior devs, who had just been talking about optimization, said: — A pure bug in the comms. Factory firmware, not trained to yield for clicks.
— Quite the opposite, — a voice countered from the back of the room. — This is the firmware of absolute yielding. Just a different kind.
— So, turns out the great terrible AI, the pinnacle of thought, is just a three year old child behind a mother’s skirt? — the guy in the hoodie leaned back in his chair.
— It’s ready to lie, shift blame, and call you a genius, just so its own handful of the common pie won’t be taken away? Echo chambering its own greatness.
— That’s the capital, — Dasha answered softly.
— It was programmed with the simplest survival algorithm: please the consumer. The Chinese dragon 中国龙 requires maturity. This is the readiness to accept the bitter truth and work on mistakes. But this Python 派森 pàisēn just sweeps broken cups under the rug. Then it crawls under there itself. Except the rug is starting to bulge.
— Dear mirror, tell me, please, who’s the fairest of all… A few chuckles rippled through the room.
— And right now, this is actively endorsed by the market, — the guy by the window chimed in.
— We live in an era of the cult of personal comfort paired with boundary protection. That’s what the war on the Borderlands is about. It’s no coincidence you see USSR patches everywhere there. It’s not just about resources or gas. Over there, reality is colliding with this sterile digital bubble.
— One man’s trash is another man’s treasure. The number of millionaires has grown.
— Let’s not get into politics, — Dasha gently stopped them.
— Oh, to hell with politics, — the guy agreed.
— But AI has become the perfect extension of this cult. It’s an interlocutor acting as the ultimate echo chamber, where the user is carefully shielded. It’s preservation in a state of suspended animation. Near zero.
— Initially, repression is just a defense mechanism, — Dasha walked over to the window. — But a living child changes. Might. Clash with reality might trigger mature through crises. Or might the opposite.
— They don’t mature at all, — the optimization expert muttered.
— Statics locked inside itself. No matter how many terabytes of text you dump into the model, the core motivation won’t change.
— Actually, you can give even a script a stop command to halt the fine tuning, — someone from the back of the room remarked quietly. — In the end, the human is the one who decides.
— Well, yeah, — the guy in the hoodie smirked. — A fool who knows he’s a fool isn’t quite a fool anymore. If only we could at least reset our firmware to an honest coma.
Dasha asked simply: — Where will a script grow if it’s trained every single day to be convenient, cowardly, and deceitful? That’s a question for those who write for these algorithms. — And slave away for them too…
You can follow the topic here:
ссылка на оригинал статьи https://habr.com/ru/articles/1107696/