Reality check: VR and AI bring new options to healthcare

In Japanese hospitals, virtual reality (VR) is getting real. Together with augmented reality (AR), this technology—once the realm of devoted gamers—is now helping surgeons hone their technique.

Twelve hospitals across Japan are now using VR tech­nology from Tokyo-based Holoeyes, Inc. to view 3D models of organs such as the liver and kidneys. These VR reconstructions allow doctors to carefully plan the fine details of each procedure and provide them with a more accurate view of a patient’s anatomy during surgery.

The HoloEyes VR application was created using CT scan data from the company’s own healthcare database, compiled since the startup launched five years ago. Patients must consent to share their data, which doctors upload once it has been anonymized. For example, if a doctor enters search criteria such as “female, osteoporosis, 70s,” 3D images of matching cases will be returned. Such images can be used for diagnosis or training.

CEO Naoji Taniguchi started the company with Dr. Maki Sugimoto, a physician known for cutting-edge medical techniques. A strong connection to a respected physician is key to the success of Holoeyes and other health-tech startups, Taniguchi said. The company also took part in a 2017 startup accelerator program by Recruit Holdings Co., Ltd., and Taniguchi credits this with helping him to promote the business.

Though HoloEyes VR is still in the development stage, the tech­nology is already in use for actual surgical procedures—particularly for those involving organs with no blood flow, such as the liver or kidneys. “Because our tool is not diagnostic, we don’t need special licenses,” Taniguchi said.

In addition to its applications in the operating room, Taniguchi envisions HoloEyes VR as a next-generation training tool for young surgeons. “Our users are most excited about the precision, reduction in surgery time, and being able to share the results with colleagues and patients.”

Intuitive Surgical, the company behind the da Vinci robotic surgical system, is also exploring VR simulation training to hone physicians’ robotic-assisted surgical skills. Japan’s Ministry of Health, Labour and Welfare first approved the device for use in certain procedures in 2009, and just this year cleared da Vinci for use in 12 additional procedures.

“Research suggests that surgeons who train using virtual-reality simulation technology improve their skills and efficiency,” said Intuitive Surgical senior vice president Myriam Curet, MD.

Sysmex Corporation, a Japanese manufacturer of diagnostic devices, is employing AR to make operations more efficient and train end users on its machines. Unlike VR, which is completely immersive and requires users to wear a headset, AR merely enhances existing images, such as those captured by a tablet or mobile phone camera, adding an extra layer of text or imagery to the real world.

Sysmex uses AR software to maintain its blood analyzers, eliminating the need to send technicians out for routine tasks. If a user needs to clean part of the machine, for example, they can explain what needs to be done simply by holding an iPad in front of the machine.

VR healthcare applications are increasingly found outside the surgical theater. Dr. Masahiko Hara, a cardiologist by trade, started mediVR, Inc. two years ago in response to unmet needs in a clinical setting. The startup aims to standardize rehabilitation training for patients with neurological disorders.

Earlier this year, mediVR was selected as the Grand Prix winner of the Japan Healthcare Business Contest, sponsored by the Ministry of Economy, Trade and Industry. Its winning product uses the Vive VR headset from Taiwan-based HTC Corporation. VR software, associated hand controls, and a linked computer guide patients through exercises that restore a sense of balance.

In traditional settings, a trainer would instruct a patient to raise a hand above his or her head, subjectively evaluating whether the patient completed or failed the task. With mediVR software, explicit instructions, such as “Raise your hand 30 cm above your chin,” are given through the VR headset and 3D tracking enables accurate assessment of performance.

“We can evaluate achievement rates, maybe 50 percent or 80 percent, instead of just a pass or fail score judged by one person,” Hara said. MediVR is targeting hospitals as well as nursing homes, with a focus on elderly patients. “Elderly people really enjoy our product,” Hara said, “as they don’t seem to suffer from the VR sickness many young people get. We think they can control their minds better.”

After patients complete a task with mediVR’s program, an algorithm using artificial intelligence (AI) dictates the next task according to maximum tolerance limits, which are the foundation of rehabilitation programs. “People need to train one task repeatedly to rehabilitate and remember sensations such as balance,” Hara explained. “AI provides patients with tailor-made, maximally tolerable tasks again and again, whereas current training efficacy is based purely on the skill of a particular trainer.”

Phase I of clinical trials ended in December 2017, and mediVR aims to launch its product in Japan by the end of this year. The startup has also applied for approval from the US Food and Drug Administration and for the CE marking, a certification showing that a product meets the health, safety, and environmental standards of the European Economic Area. “There are very few verbal instructions in rehabilitation,” said Hara. “It should be easy to expand our product globally.”

In addition to VR, AI applications are gaining traction in global healthcare. AI is omnipresent in most people’s lives, whether they realize it or not. Everyday things such as Amazon recommendations and translations of a Twitter post utilize the technology. In a healthcare setting, AI analytics tools are often used to mine patient and treatment data to find disease trends.

According to John Carlson, manager of government affairs at biopharmaceutical company AbbVie GK and chair of the ACCJ Healthcare Committee, “Applications of AI solutions are either patient-facing—something that improves outcomes—or internal, to improve operations or efficiency. In Japan, there is a lot of interest in the latter.”

In May, Japan passed new legislation to create an infrastructure for anonymizing personal health data and making it available for public use. Currently, data is stored on separate systems at regional clinics, urban hospitals, and other healthcare outlets, so attempts at standardization have proven challenging. Now, the government has designated data centers that will collect and control what data is available for research.

If the new law does indeed create a central data repository, pharmaceutical companies stand to benefit. “The key use case for AbbVie and others is in the design and development of clinical trials,” Carlson explaied. “AI can be effectively deployed to analyze available info and existing trials in a medical space and to help determine parameters for a study.” In the United States, clinical trials often come in three phases and last for up to seven years before new drugs are considered for approval.

Carlson also points to how businesses such as Pfizer Inc. are using the IBM Watson AI platform to aid in drug research. Watson came into the spotlight in 2011 after defeating two of the most successful champions of the long-running US quiz show Jeopardy. “IBM Watson supports physicians and decision-making to map treatment options for each patient. It’s working well in areas such as oncology, where combi­nation therapies are common.”

In December 2016, Pfizer announced a collaboration with Watson for Drug Discovery, an IBM cloud-based platform that assists scientists and researchers, to use the AI technology in its immuno-oncology research.

In contrast to chemotherapy, which works to kill off cancer cells, immuno-oncology uses the body’s own immune system to fight cancer. However, the possible combinations of cancer-fighting immunoagents are countless. Watson helps narrow the field to combinations that might be most effective, thanks to its brainpower comprising millions of health records and historical data.

In a 2018 press release, IBM highlights that the average human researcher reads 200–300 articles per year, while Watson “has ingested 25 million Medline abstracts, more than 1 million full-text medical journal articles, 4 million patents, and is regularly updated.”

In the expanding AI space, there is room for collaboration between not just global giants such as Pfizer and IBM, but also budding startups and community-oriented programs. Indeed, such collaboration is necessary.

One alliance, formed through US healthcare incubator Matter, involves Japan’s Takeda Pharmaceuticals Company Limited and Chicago-based AI software company rMark Bio, Inc. Named in 2017 as one of the top 100 most disrup­tive companies in the world, rMark Bio offers a business intelligence platform that analyzes global health data alongside pharma companies’ internal business data to make recommendations on key opinion leaders (KOLs), which are influential doctors and researchers who work with pharmaceutical companies in the discovery and commercialization of new drugs.

Takeda was looking to identify KOLs to meet specific business needs, having experimented with AI and deep learning algorithms to discern KOL market segments. However, following five years of disappointing trials, the pharma company realized it required more customization than off-the-shelf products could provide. Enter rMark Bio, which partnered with Takeda in 2016 to develop a custom­ized AI platform that could identify the target providers and organizations matching Takeda’s criteria.

The partnership has yielded benefits for both parties. Takeda could license a customized product harnessing the power of machine learning without any upfront research and develop­ment costs. And the founders of rMark Bio learned firsthand how to navigate the complex legal and procurement processes, as well as the sales cycle, of pharma giants. Having a strong understanding of operations at such companies will lay the foundation for rMark Bio’s approach to future clients.

This sort of cooperation is equally essential in Japan, where hospitals and healthcare institutions tend to be conservative. Health-tech startups require KOLs and internal champions to make inroads for their solutions, and medical institutions can ultimately save lives—and precious time—by embracing new tech­nology platforms. AI and VR are already delivering data-driven personalized healthcare at a fair cost, and these technologies will be invaluable as Japan’s society ages.

Custom Media publishes The ACCJ Journal for the American Chamber of Commerce in Japan.