New Uses for Smartphone AI: A Short Commentary on Recording History and Privacy
by Dr. Ian Cutress on September 6, 2019 7:20 AM EST- Posted in
- Smartphones
- AI
- Opinion
This opinion piece is reactionary to recent announcements.
Having just attended the Huawei keynote here at the IFA trade show, there were a couple of new features enabled through AI that were presented on stage that made the hair on the back of my neck stand on end. Part of it is just an impression on how quickly AI in hand-held devices is progressing, but the other part of it makes me think to how it can be misused.
Let me cover the two features.
"Real-Time Multi-Instance Segmentation"
Firstly, AI detection in photos is not new. Identifying objects isn’t new. But Huawei showed a use case where several people were playing musical instruments, and the smartphone camera could detect both the people from the background, and the people from each other. This allowed the software to change the background, from an indoor scene to an outdoor scene and such. What this also enabled was that individuals could be deleted, moved, or resized. Compare the title image to this one, where people are deleted and the background moved.
What does this mean? People can be removed from photos. Old lovers can be removed from those holiday photographs. Individuals can easily be removed (or added) from the historical record. The software would automatically generate the background behind them (if it’s the original background), and the size of people could even be changed. This was not only photographs, but video. The image blow shows one person increased in size, but it could just as easily be something significant.
Now I know that these algorithms already exist on photo editing software on a PC, if you know how to use it. I know that the demo that Huawei showed on stage was more of a representative aspect to AI on a smartphone, but I could imagine something similar coming to a smartphone, and being performed on a smartphone, and the goal to make it as easy to use as possible on a smartphone. How we in future might interpret the actions of our past selves (or past others) may have to take into account the level of access (and ease of use) in the ability to modify images and video.
Detecting Health Rate with Cameras
The second feature was related to Health and AR. By using a pre-trained algorithm, Huawei showed the ability for your smartphone to detect your heart rate simply by the front facing camera (and assuming the rear facing camera too). It does this by looking at small facial movements between video frames, and works on the values it predicts per pixel to get an overall picture.
Obviously, it isn’t meant to be used as a diagnostic tool (at least, I hope not). I could imagine similar technology being used with IP cameras for a home security system perhaps, and when it detects an elderly relative in distress, it can perform the appropriate action. But it lends itself to abuse, if you are able to use it on other people unsuspectingly. Does that constitute an invasion of privacy? Does it work on these smartphones with 10x zoom? I’m not sure I’m qualified to answer those questions.
A big part of me wants to see technology moving forward, with development and progression from generation to generation. But in seeing these two technology features today, there’s the tiniest part that doesn’t sit right, unless the correct security procedures are in place, such as edited images/videos have a signature marker, or only pre-registered people on a smartphone can have their heartbeat measured. Hopefully my initial fears aren't as serious as they first appear.
Facebook
Twitter
RSS
25 Comments
View All Comments
Diji1 - Friday, September 13, 2019 - link
>Then we grew to trust the written word, until the era of dominating yellow journalism, clickbait nonsense, and too-obvious propagandaWhile I think you're broadly correct, I don't think there has ever been a time when the "news" has been trustworthy. It's always been used to spread false narratives to serve the interests of the elite.
What has happened however is that the web has allowed people to see that they were full of crap whereas before they had no other frame of reference.
brucethemoose - Friday, September 6, 2019 - link
The era of digital pictures and video as reliable evidence/history is dead, but most people just don't know it yet. Ironically, going back to film might be prudent for investigators and such.Whenever I see giddiness over AI and all that extra 5G bandwidth, I all I can think "great, now my apps can burn even *more* battery and bandwidth constantly tracking me in the background." And then I could rant for days on how basically every non FOSS Android app seems to do this, when they have no reason to run in the background, and Google is perfectly content giving me 0 control over the process... But I try to restrain myself :/
vortmax2 - Friday, September 6, 2019 - link
I'd like to see some articles on progress of new battery tech for portables. The energy issue is always slow to progress, but would be nice to see some new tech on the near horizon.brucethemoose - Friday, September 6, 2019 - link
MicroLED displays, a native "dark mode," and more aggressive background process culling will easily give you all day battery life on current batteries. Beyond that, is it really a big deal?I think battery breakthroughs are more exciting for cars, wearable tech, robotics, and even laptops.
PeachNCream - Friday, September 6, 2019 - link
That last thing that gets me excited is the idea of wearing something made by or supported by Google that monitors my biometrics. Those creeps already gather far too much information.sheh - Friday, September 6, 2019 - link
I don't see heartbeat measurement as a big problem. Certainly a far lesser one than what we already have for years now: user-facing video recording, sound recording, OSes and apps with unhindered internet access by design.Valantar - Friday, September 6, 2019 - link
Great to see coverage like this - sites reporting on cutting-edge tech desperately need more and deeper commentary on both the intentions behind, possibilities of, and sociocultural ramifications of the technologies reported on. The evolution of society isn't determined by technology, but technology affects it in big ways nonetheless. As such, critical reporting and commentary is a necessity. More, please, but also thank you for this.catavalon21 - Friday, September 6, 2019 - link
+1eastcoast_pete - Sunday, September 8, 2019 - link
+1. So much of the advances in recent years have been used (squandered) on making it easier for AppleAmazonGoogleFacebook and others to "microtarget" us, rather than making us more productive, innovative or better at participating in democratic processes. If my smartphone measures my heart rate, pupillary dilation and can detect minor changes in vocalization, it's not far from being somebody's lie detector. 1984 was far too optimistic on what's possible.Threska - Monday, September 9, 2019 - link
Just think of how much of that microtargeting would have never happened if people had been willing to pay for their E-Mail, or social services. We the people have had as much a hand in our own demise by our desires for "frugal" things as anything the big boys have done.