Isaac Asimov was right. The dangers of artificial intelligence that he foreshadowed in his collection of short stories, I, Robot, are coming true today, most likely in the halls of your kids’ school.
You may have heard of the app that’s causing me to lose sleep: SimSimi (pronounced Shim Shimmy). The developer in Korea, who actually designed the program over 10 years ago, describes the app as an artificial intelligence conversation program. Basically, the app is a robotic text buddy. In the app, you text SimSimi and SimSimi replies. The concept is a bit like Siri or Google Now, without the personal-assistant spin. In other words, SimSimi has a huge database of prepared responses to your questions and comments, much like Siri does on your iPhone. So, when you ask, “What should I have for lunch?” you get an immediate response, “Hmmm, I think you should have a peanut butter and jelly sandwich!”
At first blush, the app seems pretty innocent. But here’s where SimSimi differs from Siri—SimSimi can learn from its users. If a user asks a question about the best place to eat in Hoboken, NJ, and SimSimi doesn’t have an answer in its database, it would respond, “I don’t know how to respond.” The user could then tap on the “Teach” feature and educate SimSimi. The user fills out a box with the query, “When someone says this,” with his question or phrase, in this case: “Best place to eat in Hoboken, NJ.” He then fills out the space that says, “SimSimi may respond with this,” with the response he wants the system to learn, in this case the restaurant he thinks is best.
However, some users are less civically-minded than others. Their sense of what’s funny might lead them to write, “Hoboken is an armpit…who would want to eat there! Gross!”
You’ve probably already begun to see why SimSimi is a school’s worst nightmare. Students in your kids’ school probably use SimSimi, especially if they’re in middle or high school. They chat with it for fun to see what hilarious responses to their questions SimSimi might come up with. It’s a version of a game you’ve probably played with your friends and Siri. However, your kids have probably also figured out that they can ask SimSimi about their classmates and their teachers. Since the student “Miriam Steinbergfeld” and teacher “Mrs. Cohen” aren’t known to SimSimi, no automatic response will be generated. One of your kids’ friends might think it’s funny to “teach” SimSimi about Miriam and Mrs. Cohen. That same student might not be very friendly with or fond of either and might chose to teach SimSimi that Miriam is a big loser and the Mrs. Cohen smells like onions (only, kids’ comments will likely be more ribald and offensive). The next day, when Miriam and her friends are playing with SimSimi at lunch, her friends might ask SimSimi about Miriam. No longer will SimSimi be ignorant about Miriam. This time she’ll respond that “Miriam is a big loser!” Prepare for tears and drama.
It goes without saying that SimSimi is the latest and most pernicious in a line of tools used for cyber-bullying. It’s especially dangerous because SimSimi is committed to protecting and making invisible the identity of any person who has ever taught SimSimi anything. So, the student who taught SimSimi that Mrs. Cohen smells like onions fears no recourse since her activity on the site can never be uncovered or revealed.
SimSimi has a disclaimer that pops up when you first attempt to teach it anything. The disclaimer says, “You agree not to teach messages (dialogue sets) that harass, abuse, defame, or otherwise infringe on any other party and understand that by doing so, you may be subject to criminal penalties.” Clearly, users are not listening.
It’s imperative that parents, teachers, and school leaders become aware of this app and its potential impact on the school community. I don’t have any good solutions for how to address the issue. Talking about it with your kids might backfire and increase usage; people who didn’t know about it might become interested when they hear about it from you, especially when they realize that it’s totally anonymous and untraceable. At the same time, we can’t stand idly by while the abuse goes on. Teaching our kids about the value of speaking kindly, especially through digital media, must be a priority.
Rabbi Maccabee Avishur is the Associate Director for Teaching and Learning at Yeshiva University’s Institute for University-School Partnership. He can be reached at avishur_yu.edu.
by Rabbi Maccabee Avishur
