Augmented Reality
Assisted Manufacturing

Guiding assembly operators with projected work instructions

For my study in Human Computer Interaction and Design I did my Master’s thesis at Scania, the biggest truck manufacturer in Europe. The department is called the Smart Factory Lab, it is Scania’s R&D for production and logistics. After this project, I started working for this same department. Initially, to continue developing the system that I created that resulted from my thesis project, and leading the project to implement it. For obvious IP protection reasons I cannot show the system in its current state but I can share the results of the thesis project, since these are also published in my paper. You can read the paper if you want to know more about the user studies I did to evaluate the system, here I’ll just show the technology.

The Problem

At a certain station in the Scania truck production end assembly line, operators need to mount press screws in the chassis frame of the truck. These screws are used to mount other parts later down the line. Scania offers full customization for new ordered trucks, resulting in almost endless different press screw configurations. The work instructions are printed on paper, and only include which screws need to be mounted for what kind of part. The operators need to learn by heart where to mount the screws in the frame. This process obviously is highly sensitive for human errors and takes a long time to learn. When the operator doesn’t know what to do, assistance is needed which takes a long time and often results in a temporary stop of the assembly line. Every second the line is not running, a lot of money is wasted. Next to the business risk, this also causes a high mental workload for the operators.

The Solution

My idea was to solve the problem with a custom interactive light guidance installation, paired with an application on a big touch screen for monitoring and user input. Light guidance is a form of (industrial) AR that is unubtrusive for the operators, other than for example AR goggles that are impractical for such an industrial environment. The installation projects the work instructions directly on the boxes containing the screws to show which screws are needed and how many, and on the frame to show exactly where the screws need to be mounted. Every screw different kind of screw is indicated with a different symbol. On the screen, the operators can get more information about the current chassis and the parts that need to be mounted, and they can set their preferences for things like the kind of visualizations and the way of interaction.

The learning time and amount of human errors will drastically decrease since now the operators will only have to follow the light instructions. The stopping time of the assembly line will decrease since no time is lost anymore with waiting for assistance. And the mental workload will decrease since people will not have to learn everything by heart and are less afraid of making errors.

The Proof of Concept

The Prototype

Based on the concept described above I created a prototype to do user tests and see if the concept would actually work. This was done in the Smart Factory Lab, and mimiced the real life station in production. The prototype was given the name Max Augmented Reality (MAXAR). In my research paper I speak of Augmented Reality Assisted Manufacturing (ARAM), a general name for AR applications in the manufacturing industry that my mentor and I thought of ourselves (there was no name for it yet and ARAM just sounds good). So, it would be great if you can call it ARAM as well next time you talk about it with someone, so the name gets adopted haha!

For the prototype we got a real chassis frame and made a construction with the boxes containing the screws above the frame, just as in the real station. A short throw projector, placed on the black pole, covers the frame and the boxes. Because of the wide projection angle, the projecter can be very close to the frame (50 cm) and there is no occlusion when the user is standing in front of the frame. Since there is a big tool board that has a lot of unused space, the prototype uses this space to show extra information.

A big touchscreen is placed next to the frame that runs the application for user input. The application shows all the parts that need to be mounted and the location on the frame. The user is able to select a part and the application will show only the locations of the screws for that particular part. The application is web based, so it could also be monitored and controlled remotely. This same applications also generates the light animations. It generates the instructions automatically with every new chassis, so they are not put in manually like what needs to be done with existing light guidance installations for manufacturing on the market. Putting in the instructions manually wouldn’t be an option because of the many different possible configurations. The main challenge of the application was to integrate several existing software systems that are used by Scania containing the necessary data to be able to create the right assembly instructions.

Since it was projecting a 2D image on a 3D surface, there needed to be a calibration mode to be able to project the right instructions on the right place. This was also controllable through the touch screen. With data from the CAD model of the truck, the system is able to locate all the holes in the frame.

The evaluation

To test the concept, we did qualitative and quantitative user tests with both experienced operators and people that have never worked at this station. We compared the current system (called MONA instructions) with the MAXAR prototype. The people without experience participated in the quantitative study and were asked to follow the assembly process of mounting press screws, either with the MONA instructions or the prototype. Both groups were participating in the qualitative study in which they were asked to follow the assembly process and fill out a questionnairre after.

Test participants had a maximum amount of time to finish the test. With the current system almost nobody finished in time. With the new system everyone did.

There was a big difference in the duration. As described above, the people testing the current system needed all the time they were given (only one finished just in time). The other group needed less time already the first trial, and there was a steep learning curve.

The current system had a worrisome average accuracy. The accuracy of the new system was significantly higher and indicating that the system can be used in production (with the accepted error margin) operators without any prior education.

Both systems were evaluated during the System Usability Score. The MONA instructions score way below the minimum and thus has a very poor perceived usability. The MAXAR system on the other hand scored very high, indicating that it is very usable and intuitive to use.

From open questionnaire answers and observed remarks of the test users a sentiment anylisis was made. The users show a large negative sentiment towards the current system and a moderate positive sentiment towards the new system.

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from Youtube
Consent to display content from Vimeo
Google Maps
Consent to display content from Google