AI is a toy or a life ?
My Friend Cayla’ is an internet-connected doll that uses voice recognition technology to chat and interact with children in real time. Cayla’s conversations are recorded and transmitted online to a voice analysis company.
This raised concerns that hackers might spy on children or communicate directly with them as they play with the doll. There are also concerns about how kids’ voice data is used. In 2017 German regulators urged parents to destroy the doll, classifying it as an “illegal espionage apparatus”.
Cayla is just one example of a new wave of artificial intelligence (AI) toys that “befriend” children. Manufacturers often claim they are educational, enhancing play and helping children develop social skills. But consumer groups warn that smart toys, like other “things” we connect to the internet, might put security and privacy at risk.
The company records and collects all the child’s conversations with the toy, and possibly those with other children and adults who also interact with it. The company is probably storing this data and certainly using it to create a better product.
The location of the toy affects how the data is stored. For example, in the US, companies creating educational toys can store data for longer than other companies. So when the manufacturer describes their toy as educational, it opens up that right to hold on to the data for longer.
As more devices – many marketed as educational toys – come onto the market, they are setting off alarm bells around privacy, bias, surveillance, manipulation, democracy, transparency and accountability.
Most economists would argue that improving and increasing access to education is one of the best ways to close the gap between the developing and developed world. AI-enabled educational toys and “teachers” could make a hugely beneficial difference in the developing world.
But, if venture capitalist and former Google China CEO Kai Fu Lee is correct, the data collected from these devices would simply be used by the big AI companies in the West and China, rather than benefiting children, their parents or the countries in which they live.
As well as the risk of hacking, we also need to think about what these toys are saying to our children. Who is the arbiter of these conversations? Who coded the algorithms (their unintended biases could creep in)? Do the values the child is being exposed to align with those of the parents? Will parents be able to choose the values the toy is coded with?
If the toy is educational, is the algorithm being checked by someone who is at least qualified to teach?
These toys will be very influential because the children will be conversing with them all the time. For example, if the doll says it is cold and the child asks his or her parents to buy it a coat, is that advertising?
If the child has a toy that can talk back, instead of an invisible friend, will this affect creative play for good or ill? The child won’t have to invent stories about an invisible friend anymore. Could this huge change in creative play alter us as human beings?
If data is being collected, even if it isn’t being stored, does the company have a duty to “red flag” children who share suicidal thoughts or other self-harming behaviour? What if the child confides in the toy that he or she is being abused, will the company report this to the relevant authorities? And then what will the company do with that information?
Parents need to have answers to these questions before they buy the devices. At the very least, they should check that their child is learning values from AI toys that concur with their own. The problem, though, with governance of smart toys is that the AI is learning and changing with each interaction with the child.
This doesn’t mean we’re saying that AI-enabled toys are bad. They might one day help us achieve precision learning (using AI to tailor education to each child’s needs). And AI toys could be excellent for preparing children to work alongside autonomous robots. What we are saying is that children are vulnerable and so we should consider now how AI is used around them and not beta test it on them.