Emotional Vulnerability and AI Girlfriends: A Gender Perspective

An official order from the Italian data Protection Authority has recently secured A.I. powered chat-box Replika on the ground that it increases risks to emotionally vulnerable people. Replika is part of the fast-growing generative AI companion market, where users (mostly men, as statistics show) customize everything about their virtual partners (women), from appearance and personality to the content and modalities of their relationship. This paper analyzes the twofold vulnerability arising from using AI companions. The first profile, at the micro level, deals with the creation of a virtual girlfriend, in a one-sided relationship, that can lead to or reinforce abusive verbal behaviors. The second profile, at the macro level, concerns the more general aspect that AI companions amplify harmful stereotypes and biases against women and girls. Users’ violence, even when carried out on a cluster of code, reflect the broader societal problem of unacceptably high levels of violence against women and girls across all contexts. Moreover, the relevance of this topic is further confirmed by the increasing number of film productions (think of the movie "Her" or of the TV series on demand, among others). While the actual trend shows an increasing and growing – if not yet sufficient – commitment by the institutions at national and international level towards pathways based on education on affectivity, respect and dignity of the person, the virtual dimension seems to show an exactly opposite trend, with the concrete risks that abusive behaviors move back again to the real world.