Monthly Archives: March 2017

Memristor chips that see patterns over pixels

Faster image processing could have big implications for autonomous systems such as self-driving cars, says Wei Lu, U-M professor of electrical engineering and computer science. Lu is lead author of a paper on the work published in the current issue of Nature Nanotechnology.

Lu’s next-generation computer components use pattern recognition to shortcut the energy-intensive process conventional systems use to dissect images. In this new work, he and his colleagues demonstrate an algorithm that relies on a technique called “sparse coding” to coax their 32-by-32 array of memristors to efficiently analyze and recreate several photos.

Memristors are electrical resistors with memory — advanced electronic devices that regulate current based on the history of the voltages applied to them. They can store and process data simultaneously, which makes them a lot more efficient than traditional systems. In a conventional computer, logic and memory functions are located at different parts of the circuit.

“The tasks we ask of today’s computers have grown in complexity,” Lu said. “In this ‘big data’ era, computers require costly, constant and slow communications between their processor and memory to retrieve large amounts data. This makes them large, expensive and power-hungry.”

But like neural networks in a biological brain, networks of memristors can perform many operations at the same time, without having to move data around. As a result, they could enable new platforms that process a vast number of signals in parallel and are capable of advanced machine learning. Memristors are good candidates for deep neural networks, a branch of machine learning, which trains computers to execute processes without being explicitly programmed to do so.

“We need our next-generation electronics to be able to quickly process complex data in a dynamic environment. You can’t just write a program to do that. Sometimes you don’t even have a pre-defined task,” Lu said. “To make our systems smarter, we need to find ways for them to process a lot of data more efficiently. Our approach to accomplish that is inspired by neuroscience.”

A mammal’s brain is able to generate sweeping, split-second impressions of what the eyes take in. One reason is because they can quickly recognize different arrangements of shapes. Humans do this using only a limited number of neurons that become active, Lu says. Both neuroscientists and computer scientists call the process “sparse coding.”

“When we take a look at a chair we will recognize it because its characteristics correspond to our stored mental picture of a chair,” Lu said. “Although not all chairs are the same and some may differ from a mental prototype that serves as a standard, each chair retains some of the key characteristics necessary for easy recognition. Basically, the object is correctly recognized the moment it is properly classified — when ‘stored’ in the appropriate category in our heads.”

Similarly, Lu’s electronic system is designed to detect the patterns very efficiently — and to use as few features as possible to describe the original input.

In our brains, different neurons recognize different patterns, Lu says.

“When we see an image, the neurons that recognize it will become more active,” he said. “The neurons will also compete with each other to naturally create an efficient representation. We’re implementing this approach in our electronic system.”

The researchers trained their system to learn a “dictionary” of images. Trained on a set of grayscale image patterns, their memristor network was able to reconstruct images of famous paintings and photos and other test patterns.

Machine learning create hyper predictive computer models

Drug development is a costly and time-consuming process. To narrow down the number of chemical compounds that could be potential drug candidates, scientists utilize computer models that can predict how a particular chemical compound might interact with a biological target of interest — for example, a key protein that might be involved with a disease process. Traditionally, this is done via quantitative structure-activity relationship (QSAR) modeling and molecular docking, which rely on 2- and 3-D information about those chemicals.

Denis Fourches, assistant professor of computational chemistry, wanted to improve upon the accuracy of these QSAR models. “When you’re screening a set of 30 million compounds, you don’t necessarily need a very high reliability with your model — you’re just getting a ballpark idea about the top 5 or 10 percent of that virtual library. But if you’re attempting to narrow a field of 200 analogues down to 10, which is more commonly the case in drug development, your modeling technique must be extremely accurate. Current techniques are definitely not reliable enough.”

Fourches and Jeremy Ash, a graduate student in bioinformatics, decided to incorporate the results of molecular dynamics calculations — all-atom simulations of how a particular compound moves in the binding pocket of a protein — into prediction models based on machine learning.

“Most models only use the two-dimensional structures of molecules,” Fourches says. “But in reality, chemicals are complex three-dimensional objects that move, vibrate and have dynamic intermolecular interactions with the protein once docked in its binding site. You cannot see that if you just look at the 2-D or 3-D structure of a given molecule.”

In a proof-of-concept study, Fourches and Ash looked at the ERK2 kinase — an enzyme associated with several types of cancer — and a group of 87 known ERK2 inhibitors, ranging from very active to inactive. They ran independent molecular dynamics (MD) simulations for each of those 87 compounds and computed critical information about the flexibility of each compound once in the ERK2 pocket. Then they analyzed the MD descriptors using cheminformatics techniques and machine learning. The MD descriptors were able to accurately distinguish active ERK2 inhibitors from weakly actives and inactives, which was not the case when the models used only 2-D and 3-D structural information.

“We already had data about these 87 molecules and their activity at ERK2,” Fourches says. “So we tested to see if our model was able to reliably find the most active compounds. Indeed, it accurately distinguished between strong and weak ERK2 inhibitors, and because MD descriptors encoded the interactions those compounds create in the pocket of ERK2, it also gave us more insight into why the strong inhibitors worked well.

“Before computing advances allowed us to simulate this kind of data, it would have taken us six months to simulate one single molecule in the pocket of ERK2. Thanks to GPU acceleration, now it only takes three hours. That is a game changer. I’m hopeful that incorporating data extracted from molecular dynamics into QSAR models will enable a new generation of hyper-predictive models that will help bringing novel, effective drugs onto the market even faster. It’s artificial intelligence working for us to discover the drugs of tomorrow.”

Faster technique to remotely operate robots

The traditional interface for remotely operating robots works just fine for roboticists. They use a computer screen and mouse to independently control six degrees of freedom, turning three virtual rings and adjusting arrows to get the robot into position to grab items or perform a specific task.

But for someone who isn’t an expert, the ring-and-arrow system is cumbersome and error-prone. It’s not ideal, for example, for older people trying to control assistive robots at home.

A new interface designed by Georgia Institute of Technology researchers is much simpler, more efficient and doesn’t require significant training time. The user simply points and clicks on an item, then chooses a grasp. The robot does the rest of the work.

“Instead of a series of rotations, lowering and raising arrows, adjusting the grip and guessing the correct depth of field, we’ve shortened the process to just two clicks,” said Sonia Chernova, the Georgia Tech assistant professor in robotics who advised the research effort.

Her team tested college students on both systems, and found that the point-and-click method resulted in significantly fewer errors, allowing participants to perform tasks more quickly and reliably than using the traditional method.

“Roboticists design machines for specific tasks, then often turn them over to people who know less about how to control them,” said David Kent, the Georgia Tech Ph.D. robotics student who led the project. “Most people would have a hard time turning virtual dials if they needed a robot to grab their medicine. But pointing and clicking on the bottle? That’s much easier.”

The traditional ring-and-arrow-system is a split-screen method. The first screen shows the robot and the scene; the second is a 3-D, interactive view where the user adjusts the virtual gripper and tells the robot exactly where to go and grab. This technique makes no use of scene information, giving operators a maximum level of control and flexibility. But this freedom and the size of the workspace can become a burden and increase the number of errors.

The point-and-click format doesn’t include 3-D mapping. It only provides the camera view, resulting in a simpler interface for the user. After a person clicks on a region of an item, the robot’s perception algorithm analyzes the object’s 3-D surface geometry to determine where the gripper should be placed. It’s similar to what we do when we put our fingers in the correct locations to grab something. The computer then suggests a few grasps. The user decides, putting the robot to work.

“The robot can analyze the geometry of shapes, including making assumptions about small regions where the camera can’t see, such as the back of a bottle,” said Chernova. “Our brains do this on their own — we correctly predict that the back of a bottle cap is as round as what we can see in the front. In this work, we are leveraging the robot’s ability to do the same thing to make it possible to simply tell the robot which object you want to be picked up.”

By analyzing data and recommending where to place the gripper, the burden shifts from the user to the algorithm, which reduces mistakes. During a study, college students performed a task about two minutes faster using the new method vs. the traditional interface. The point-and-click method also resulted in approximately one mistake per task, compared to nearly four for the ring-and-arrow technique.

SMS texting could help cut internet energy use

Researchers looking more closely than ever before into everyday mobile device habits — and in particular the impact smartphone and tablet apps have on data demand — are suggesting ways that society can cut back on its digital energy consumption.

European smartphone data growth is relentless, with data traffic predicted to rise from 1.2GB to 6.5GB a month per person. Although precise energy estimates are difficult, and depend on the service and network conditions, for video streaming each gigabyte of data can be estimated to consume 200 watt-hours of energy through Internet infrastructure and datacentres.

Following a detailed study on Android device users, and comparing observations with a large dataset of almost 400 UK and Ireland mobile devices, computer scientists at Lancaster University and the University of Cambridge identified four categories of data-hungry services — watching video, social networking, communications and listening. These four categories equate to around half of mobile data demand.

Watching videos (21 per cent of daily aggregate mobile data demand) and listening to music (11 per cent) are identified as the two most data-intensive activities. They are popular during the peak electricity demand hours of between 4pm and 8pm (when carbon emissions due to generation on UK National Grid are highest), but watching is particularly popular in the late hours before bedtime. People make their workdays more enjoyable by streaming music, with listening demand peaking during commuting hours and also at lunchtime.

The researchers recommend designers look at creating features for devices or apps that encourage people to gather together with friends and family to enjoy streamed media — reducing the overall number of data streams and downloads.

Kelly Widdicks, PhD student at Lancaster University’s School of Computing and Communications, said: “To reduce energy consumption, app designers could look at ways to coordinate people to enjoy programmes together with friends and family, listening to locally-stored or cached music, and developing special celebratory times — weekly or monthly — to more fully appreciate streamed media, rather than binge watching.

“But at least equally important is the role of service and content providers. Our studies show significant evidence that automatic queueing of additional video, and unprompted loading of selected content leads to more streaming than might otherwise have happened.”

Social networking also causes large demands for data and the researchers suggest systems architects re-evaluate the social importance and meaning of videos streamed over these platforms, alongside the energy required.

“Media, and therefore data demand, is embedded into social networking apps. We propose that this dogma of all-you-can-eat data should be challenged, for example by reducing previews or making people click through to view content that interests them,” said Dr Oliver Bates, senior researcher at Lancaster University’s School of Computing and Computing and Communications. “This may dissuade people from simply viewing media just because it is easily accessible.

“Our participants indicated there was often little meaning or utility to the automated picture feeds, and video adverts common to many social media apps.”

Figures obtained through the study indicate energy consumption through instant messaging apps’ data demand is around ten times higher than that of SMS. If people were to default back to sending messages via SMS rather than instant messaging services, it would help to reduce data and energy consumption further. However, the researchers point out that to make an SMS-like service practical again, more of the features of instant messages would need to be adopted.

“By using SMS for simple text messaging, or a low-overhead instant messaging service (such as one which sends images at lower resolutions), there is good potential to decrease energy consumption from communications,” said Miss Widdicks.

“Mobile service providers and device designers can make it more convenient for people to switch between communication methods. SMS and MMS services could be revised to better suit the phone user today, such as by sending photos at a lower cost to the subscriber, catering better for group messages and by informing users that their sent messages have been received — all reasons why people use instant messaging apps,” she added.

How to catch a phisher

Computer science professors Rakesh Verma, Arjun Mukherjee, Omprakash Gnawali and doctoral student Shahryar Baki used publicly available emails from Hillary Clinton and Sarah Palin as they looked at the characteristics of phishing emails and traits of the email users to determine what factors contribute to successful attacks. The team used natural language generation — a process used to replicate human language patterns — to create fake phishing emails from real emails. It’s a tactic used by hackers to execute “masquerade attacks,” where they pretend to be authorized users of a system by replicating the writing styles of the compromised account.

Using the Clinton and Palin emails, the research team created fake emails and planted certain signals, such as fake names, repetitive sentences and “incoherent flow.” Study participants were then given eight Clinton emails and eight Palin emails — four were real, four were fake. Volunteers were asked to identify which emails were real and explain their reasoning. The study took into account the reading levels of the Clinton and Palin emails as well as the personality traits, confidence levels and demographics of the 34 volunteers who participated.

The results of the study showed that:

  • Participants could not detect the real emails with any degree of confidence. They had a 52 percent overall accuracy rate.
  • Using more complex grammar resulted in fooling 74 percent of participants.
  • 17 percent of participants could not identify any of the signals that were inserted in the impersonated emails.
  • Younger participants did better in detecting real emails.
  • Only 50 percent of the participants mentioned the fake names.
  • Only six participants could show the full header of an email.
  • Education, experience with emails usage and gender did not make a difference in the ability to detect the deceptive emails.

“Our study offers ideas on how to improve IT training,” Verma said. “You can also generate these emails and then subject the phishing detectors to those kind of emails as a way to improve the detectors’ ability to identify new attacks.”

In the case of the recent Google Docs attack, Verma says people fell for the scam because they trust Google. When users opened the given URL, they were sent to a permissions page and hackers got control of their emails, contacts and potentially their personal information. Google stopped the scam, removed the fake pages and disabled offending accounts. Verma said a real Google Docs application will generally not ask for permission to access your contacts or read your emails.

The “WannaCry” ransomware attack that has hit banks, hospitals and government agencies around the globe is also spread through email phishing and can be spread through the Google Doc-type “worm” as well.

What all email users need to know in order to protect themselves:

  • Look closely at the sender of the email and the full header that has information about how the email was routed.
  • Look at the body of the email for any fake, broken links that can be identified by hovering a mouse over them.
  • Think about the context of the email and how long it has been since you have had contact with the sender.

“There will be copycat attacks in the future and we have to watch out for that,” said Verma.

Delineates breast cancers on digital tissue slides

Looking closer, the network correctly made the same determination in each individual pixel of the slide 97 percent of the time, rendering near-exact delineations of the tumors.

Compared to the analyses of four pathologists, the machine was more consistent and accurate, in many cases improving on their delineations.

In a field where time and accuracy can be critical to a patient’s long-term prognosis, the study is a step toward automating part of biopsy analysis and improving the efficiency of the process, the researchers say.

Currently, cancer is present in one in 10 biopsies ordered by physicians, but all must be analyzed by pathologists to identify the extent and volume of the disease, determine if it has spread and whether the patient has an aggressive or indolent cancer and needs chemotherapy or a less drastic treatment.

Last month, the U.S. Food and Drug Administration approved software that allows pathologists to review biopsy slides digitally to make diagnosis, rather than viewing the tissue under a microscope.

“If the network can tell which patients have cancer and which do not, this technology can serve as triage for the pathologist, freeing their time to concentrate on the cancer patients,” said Anant Madabushi, F. Alex Nason professor II of biomedical engineering at Case Western Reserve and co-author of the study detailing the network approach, published in Scientific Reports.

The study

To train the deep-learning network, the researchers downloaded 400 biopsy images from multiple hospitals. Each slide was approximately 50,000 x 50,000 pixels. The computer navigated through or rectified the inconsistencies of different scanners, staining processes and protocols used by each site, to identify features in cancer versus the rest of the tissue.

The researchers then presented the network with 200 images from The Cancer Genome Atlas and University Hospitals Cleveland Medical Center. The network scored 100 percent on determining the presence or absence of cancer on whole slides and nearly as high per pixel.

“The network was really good at identifying the cancers, but it will take time to get up to 20 years of practice and training of a pathologist to identify complex cases and mimics, such as adenosis,” said Madabhushi, who also directs the Center of Computational Imaging and Personalized Diagnostics at Case Western Reserve.

Network training took about two weeks, and identifying the presence and exact location of cancer in the 200 slides took about 20 to 25 minutes each.

That was done two years ago. Madabhushi suspects training now — with new computer architecture — would take less than a day, and cancer identification and delineation could be done in less than a minute per slide.

“To put this in perspective,” Madabhushi said, “the machine could do the analysis during ‘off hours,’ possibly running the analysis during the night and providing the results ready for review by the pathologist when she/he were to come into the office in the morning.”