Sign In
Information about science based innovations, solid facts to support decision-making, performance and business development for VTT’s partners, customers and anyone interested in high technology and its applications.
A magazine on science, technology and business

​The helmet display provides the user with step-by-step instructions on how to do the required work in the right order.

Maintenance support for astronauts

Text: Matti Välimäki | Photo: Timo Porthan | 14.12.2017

An augmented reality tool provides support in critical tasks that need to be done quickly and correctly first time. VTT's tool for space stations has demonstrated its power in the Columbus Laboratory in Germany.

​An international project led by VTT has developed a new augmented reality (AR) tool for the European Space Agency (ESA). Kaj Helin, a Pricipal Scientist at VTT, says that the AR tool helps in tasks such as maintenance missions that are done rarely, or that are critical to the operation of a space station.

– The system gives precise, step-by-step instructions on what to do, on a display on the astronaut's AR goggles. This reduces human errors, speeds up work and clarifies guidelines, says Helin.

For example, the user of a head-mounted display gains a clear view of where the maintenance site is located, which buttons she should press, and in what direction various levers should be moved.

– Via an IoT interface, she also receives important telemetry information, such as diagnostics of faults and information on the last maintenance round and the component lifecycle, radiation, pressure and temperature.

– The system developed through the project runs on the Microsoft HoloLens platform.

Real-time remote control also available

In practice, a critical maintenance operation on a space station can involve changing a carbon dioxide filter:

– The helmet display provides the user with step-by-step instructions on how to do the required work in the right order. It also reads elements such as QR codes and checks that the right spare part is being used. The user can also view the pressure of the air being supplied and removed.

The AR display also enables remote guidance of the maintenance person from the Earth's surface.

– In addition, the AR display view is transmitted to the Earth's surface. If an astronaut is unsure of which screw to turn, for example, the control room on Earth can use a dedicated symbol to show her.

A major breakthrough

Practical tests of the AR system for space stations have been performed in the Columbus research laboratory located in the ESA's on-ground European Astronaut Centre (EAC) in Cologne, Germany. An experienced astronaut participated in the tests.

– The results have been highly positive. ESA has been trying to get a similar system to work for ten years – but now it truly looks as though we have made a breakthrough, Helin says.

What, then, explains the success of VTT's project?

– We have a lot of expertise on AR and its possible applications, as well as strong space expertise through our partners. Another reason for the amount of progress made is the fact that AR technology has developed a great deal in recent years.

The two-year project was funded by ESA. In addition to VTT, Thales Alenia Space of Italy and the ICCS (Institute of Communications and Computer Systems) of Greece participated in the research. VTT has also been involved in the EUfunded WEKIT project, for example.

– This involves using AR for the installation of temporary goods storage on a space station. Issues such as positioning are challenging when a large, changing environment is involved.

Genuinely in space

VTT is currently engaged in negotiations on developing an AR tool prototype on Earth that is yet one step more advanced.

– Of course, the next stage is a system that truly functions on a space station in orbit.

How will data transfer actually work between Earth and an AR tool far away in space?

– One of the tricks of the system lies in keeping the amount of data between Earth and the space station small. The tool has good basic functionality and can be controlled from the ground, with very small text files transmitted to the space station via its own data channel.

On the way to Mars

Kaj Helin points out that the need for AR solutions for space craft and space stations is continuously growing. Demand is being increased by the fact that humankind is now truly on its way to Mars.

– Training is one of the challenges involved in conquering Mars. For example, when an astronaut spends half a year on a space station, she needs two years of advance training. Advance training on the same basis, i.e. ten years of training for a trip to Mars (where the trip would take a couple years) is an unrealistic idea. Some training will have to be done during the trip, using AR systems.

The greater general interest in space also bodes well for AR systems. In addition, both space stations and mines are being planned for the moon.

Many applications on Earth as well

On the other hand, AR tools are like many other types of space technology: plenty of applications can be found for space solutions on Earth.

– The methods developed through our project can also be used in, say, power plants and on ships – or in the aviation industry, house building and in machine building. They tend to be suitable for critical locations, where there is no room for error or, due to costs, measures must be taken quickly and correctly first time.

Helin believes that AR systems will soon become more common on Earth and will be far advanced within two or three years' time.

– Many companies are engaged in testing. The technology has reached a sufficiently advanced stage. The current issue is mainly about how to create system content cost-effectively.

 

Is maintenance becoming knowledge work?

AR (Augmented Reality), VR (Virtual Reality) and digitalisation will change maintenance work in general.

During VTT's project in the Dimecc S-Step programme, it was observed that the new technology is helpful in preparing for maintenance tasks as a precise idea of the situation at the site is obtained in advance. This will also support problem solving on site and can provide concrete guidance for work tasks. The new technology also streamlines reporting if, for example, images and sound recordings of the site are automatically linked to the information system.

– The idea of remote maintenance engineers was also an S-Step theme. In such a case, a person on site does the work under the guidance of another, more experienced specialist, via an AR tool. This is, of course, the easiest way to get started with AR solutions, since it does not require separate AR content production, says Eija Kaasinen, Principal Scientist at VTT.

Access to the relevant information

VTT is currently leading a research project, called Dynavis, which is a continuation of the S-Step work. Among other things, the project explores how visual information can be provided in the most sensible manner to users of the AR helmet, for example.

– A huge amount of data is available. Maintenance instructions, monitored data as well as information and tips shared by other users can be useful to maintenance engineers; AR and the social media can be combined. The challenge is how, say, a user of an AR helmet can easily find out what information is available and obtain the most relevant information.

In addition to Tekes, the Dynavis project is being funded by KONE, Konecranes, Wärtsilä, Eeedo and 3D Studio Blomberg. The TAUCHI Research Group of the University of Tampere is involved in the research, alongside VTT. Dynavis entity, which includes also companies' own development projects, is being coordinated by Dimecc.

Working practices must also be developed

Eija Kaasinen points out that a new maintenance culture cannot be developed only with technology push.

– New tools and new working practices must be developed in parallel and in cooperation with maintenance personnel, she stresses.

So, when does Kaasinen think that maintenance personnel using AR helmets will be an everyday sight in places such as factories?

– This could happen fairly soon. But I think that AR glasses and helmets will only be used occasionally at first, according to need. Development is needed in the devices' ergonomics. Long-term use of AR glasses would be too cumbersome at the moment, muses Kaasinen.

 

 

VTT Impulsehttps://www.vttresearch.com/ImpulseVTT Impulse
A birthday gift from us to ourselveshttps://www.vttresearch.com/Impulse/Pages/A-birthday-gift-from-us-to-ourselves.aspxA birthday gift from us to ourselves
A man driven by optimism and curiosityhttps://www.vttresearch.com/Impulse/Pages/A-man-driven-by-optimism-and-curiosity.aspxA man driven by optimism and curiosity
An export product tougher than nailshttps://www.vttresearch.com/Impulse/Pages/An-export-product-tougher-than-nails.aspxAn export product tougher than nails
Matti Apunen: At the core of the solutionhttps://www.vttresearch.com/Impulse/Pages/At-the-core-of-the-solution.aspxMatti Apunen: At the core of the solution
Before information gets lost in numbershttps://www.vttresearch.com/Impulse/Pages/Before-information-gets-lost-in-numbers.aspxBefore information gets lost in numbers
Brazil paves the way for bioenergyhttps://www.vttresearch.com/Impulse/Pages/Brazil-paves-the-way-for-bioenergy.aspxBrazil paves the way for bioenergy
Business for transportationhttps://www.vttresearch.com/Impulse/Pages/Business-for-transportation.aspxBusiness for transportation
Cell screening leads to new cancer treatmentshttps://www.vttresearch.com/Impulse/Pages/Cell-screening-leads-to-new-cancer-treatments.aspxCell screening leads to new cancer treatments
Converting rags to design clotheshttps://www.vttresearch.com/Impulse/Pages/Converting-rags-to-design-clothes.aspxConverting rags to design clothes
Defence Forces seek a head starthttps://www.vttresearch.com/Impulse/Pages/Defence-Forces-seek-a-head-start.aspxDefence Forces seek a head start
Designers needed in printed technologyhttps://www.vttresearch.com/Impulse/Pages/Designers-needed-in-printed-technology.aspxDesigners needed in printed technology
Developers of the futurehttps://www.vttresearch.com/Impulse/Pages/Developers-of-the-future.aspxDevelopers of the future
Energy solutions and the 2050 goalshttps://www.vttresearch.com/Impulse/Pages/Energy-solutions-and-the-2050-goals.aspxEnergy solutions and the 2050 goals
Health benefits and efficiency without sacrificing food qualityhttps://www.vttresearch.com/Impulse/Pages/Health-benefits-and-efficiency-without-sacrificing-food-quality.aspxHealth benefits and efficiency without sacrificing food quality
How to become a leading country in flexible energy production systemshttps://www.vttresearch.com/Impulse/Pages/How-to-become-a-leading-country-in-flexible-energy-production-systems.aspxHow to become a leading country in flexible energy production systems
Intelligence for machineshttps://www.vttresearch.com/Impulse/Pages/Intelligence-for-machines.aspxIntelligence for machines
LEO brings intelligence to cylindershttps://www.vttresearch.com/Impulse/Pages/LEO-brings-intelligence-to-cylinders.aspxLEO brings intelligence to cylinders
Petri Kalliokoski: Artificial intelligence moves from words to deedshttps://www.vttresearch.com/Impulse/Pages/Petri-Kalliokoski-Artificial-intelligence-moves-from-words-to-deeds.aspxPetri Kalliokoski: Artificial intelligence moves from words to deeds
Salofa creates rapid tests for monitoring health and the environmenthttps://www.vttresearch.com/Impulse/Pages/Salofa-creates-rapid-tests-for-monitoring-health-and-the-environment.aspxSalofa creates rapid tests for monitoring health and the environment
Same old routine for the next 100 yearshttps://www.vttresearch.com/Impulse/Pages/Same-old-routine-for-the-next-100-years.aspxSame old routine for the next 100 years
Taking the bioeconomy to a new level through digitalisationhttps://www.vttresearch.com/Impulse/Pages/Taking-the-bioeconomy-to-a-new-level-through-digitalisation.aspxTaking the bioeconomy to a new level through digitalisation
Let's make bioeconomy a realityhttps://www.vttresearch.com/Impulse/Pages/Lets-make-bioeconomy-a-reality.aspxLet's make bioeconomy a reality
Open-source infrastructure in product development for the electromechanical industryhttps://www.vttresearch.com/Impulse/Pages/Open-source-infrastructure-in-product-development-for-the-electromechanical-industry.aspxOpen-source infrastructure in product development for the electromechanical industry
Clearing the way for new materials with ChemSheet technologyhttps://www.vttresearch.com/Impulse/Pages/Clearing-the-way-for-new-materials-with-ChemSheet-technology.aspxClearing the way for new materials with ChemSheet technology
Jaakko Eskola: ”The ability to reinvent yourself is vital to success.”https://www.vttresearch.com/Impulse/Pages/”The-ability-to-reinvent-yourself-is-vital-to-success”.aspxJaakko Eskola: ”The ability to reinvent yourself is vital to success.”
Intelligent transport – a trump card for Finnish exportshttps://www.vttresearch.com/Impulse/Pages/Intelligent-transport-a-trump-card-for-Finnish-exports.aspxIntelligent transport – a trump card for Finnish exports
It is the duty of entrepreneurs to seek growthhttps://www.vttresearch.com/Impulse/Pages/It-is-the-duty-of-entrepreneurs-to-seek-growth.aspxIt is the duty of entrepreneurs to seek growth
Impulse 1/2018https://www.vttresearch.com/Impulse/Pages/Impulse-1-2018.aspxImpulse 1/2018
Finnish wheels will keep on turninghttps://www.vttresearch.com/Impulse/Pages/Finnish-wheels-will-keep-on-turning.aspxFinnish wheels will keep on turning
More than siliconhttps://www.vttresearch.com/Impulse/Pages/More-than-just-Silicon.aspxMore than silicon
More speed to network services with 5Ghttps://www.vttresearch.com/Impulse/Pages/More-speed-to-network-services-with-5G.aspxMore speed to network services with 5G
Ambassadors of service-mindednesshttps://www.vttresearch.com/Impulse/Pages/Ambassadors-of-service-mindedness.aspxAmbassadors of service-mindedness
Quantum standards for the new SI systemhttps://www.vttresearch.com/Impulse/Pages/Quantum-standards-for-the-new-SI-system.aspxQuantum standards for the new SI system
Brought to you by Paptic: Wood fibre challenges plastic in carrier bagshttps://www.vttresearch.com/Impulse/Pages/Brought-to-you-by-Paptic-Wood-fibre-challenges-plastic-in-carrier-bags.aspxBrought to you by Paptic: Wood fibre challenges plastic in carrier bags
Three-dimensional virtual reality and augmented reality – Towards new worldshttps://www.vttresearch.com/Impulse/Pages/Three-dimensional-virtual-reality-and-augmented-reality-Towards-new-worlds.aspxThree-dimensional virtual reality and augmented reality – Towards new worlds
Raw material for fuels, chemicals and food from atmospheric carbon dioxidehttps://www.vttresearch.com/Impulse/Pages/Raw-material-for-fuels-chemicals-and-food-from-atmospheric-carbon-dioxide.aspxRaw material for fuels, chemicals and food from atmospheric carbon dioxide
Transport gets smarthttps://www.vttresearch.com/Impulse/Pages/Transport-gets-smart.aspxTransport gets smart
New breed of business from cellulosehttps://www.vttresearch.com/Impulse/Pages/New-breed-of-business-from-cellulose.aspxNew breed of business from cellulose
At the digital forefront of mininghttps://www.vttresearch.com/Impulse/Pages/At-the-digital-forefront-of-mining.aspxAt the digital forefront of mining
Automation gives packages wingshttps://www.vttresearch.com/Impulse/Pages/Automation-gives-packages-wings.aspxAutomation gives packages wings
The energy sector is jointly preparing for cyber threatshttps://www.vttresearch.com/Impulse/Pages/The-energy-sector-is-jointly-preparing-for-cyber-threats.aspxThe energy sector is jointly preparing for cyber threats
Digitalisation of the shipping industry requires cutting-edge researchhttps://www.vttresearch.com/Impulse/Pages/Digitalisation-of-the-shipping-industry-requires-cutting-edge-research.aspxDigitalisation of the shipping industry requires cutting-edge research
Novel Cellulose products and applicationshttps://www.vttresearch.com/Impulse/Pages/Novel-Cellulose-products-and-applications.aspxNovel Cellulose products and applications
Gerontechnology providing everyday assistance and safetyhttps://www.vttresearch.com/Impulse/Pages/Gerontechnology-providing-everyday-assistance-and-safety.aspxGerontechnology providing everyday assistance and safety
Swift and agile in business and in the foresthttps://www.vttresearch.com/Impulse/Pages/Swift-and-agile-in-business-and-in-the-forest.aspxSwift and agile in business and in the forest
Passion spurs researchhttps://www.vttresearch.com/Impulse/Pages/passion-spurs-research.aspxPassion spurs research
Cleaner city traffic with wood-based dieselhttps://www.vttresearch.com/Impulse/Pages/Cleaner-city-traffic-with-wood-based-diesel.aspxCleaner city traffic with wood-based diesel
Biobased aromatics essential to the fine chemical industryhttps://www.vttresearch.com/Impulse/Pages/Biobased-aromatics-essential-to-the-fine-chemical-industry.aspxBiobased aromatics essential to the fine chemical industry
Multifunctional polysaccharides open up new opportunities for the food processing industryhttps://www.vttresearch.com/Impulse/Pages/Multifunctional-polysaccharides-open-up-new-opportunities-for-the-food-processing-industry.aspxMultifunctional polysaccharides open up new opportunities for the food processing industry

CONTACT US

P.O. Box 1000, FI-02044 VTT, Finland
Tel. exchange +358 20 722 111
Opening hours Mon - Fri 8:00 - 16:30,
UTC +3 time zone

CUSTOMER SERVICE

info@vtt.fi
Tel. +358 20 722 7070
Opening hours Mon - Fri 9:00 - 11:00 and 12:00 - 15:00,
UTC +3 time zone