The Problem

Collaboration is never truly synchronous. Real collaboration moves fluently between different degrees of synchronous (at the same time) and asynchronous (at different times). Interruptions, for example, make synchronous work asynchronous for a short time.
While collaborative MR is becoming part of the future workplace, little research is concerned with how MR can address interruptions during collaboration.

‍

My Process

Step 1: Discovering a use case of collaborative MR

I spent the first weeks of the project interviewing and observing archeologists to learn about their work. The goal of this field research was to identify a use case for a collaborative MR environment. Archaeologists have fascinating jobs, and I found that they are highly specialized in a specific area. Which means they mostly work individually on their own data.

I did identify one promising moment of collaboration when the archeologists meet to plan the next visit to an excavation site. This is the context I used for the MR system.

Step 2: Investigating interuptions in synchronous collaboration

Another thing I learned while spending time with the archaeologists is that interruptions are expected during meetings. Making yourself a tea, a phone call coming in, a colleague knocking on the office door will all interrupt the collaboration.

Platforms for traditional remote collaboration have features like muting your microphone or turning off your camera in a video call to minimize distraction caused by such interruptions. The equivalent or alternatives to these methods in collaborative virtual environments have not been researched yet.

Here you can see what it would look like if we do not address the problem of short-term interruption:

Current Scenario: While 4 archeologists plan the next excavation in MR, a collegue enters and interrupts the discussion



press for fullscreen

In the scenario, multiple problems become evident. The most apparent problem is continuous tracking. This problem a occurs when a participant in a collaborative MR environment interacts with real-life distractions without leaving the virtual environment. The others are distracted by the private conversation, and the avatar of the absentee is in the way. So what if the others want to continue their collaboration anyway? They are now missing the absent user's expertise and consent. Additionally, when the absentee returns, the group has to stop their discussion and brief them on what has been discussed.

The following table summarizes problems and how they affect the absentee and the collaborators that stay behind:

Problems occuring because of interuption



press for fullscreen

All these problems disturb the flow of collaboration, and I designed a concept for fixing them.

Step 3: Creating the concept of the Absence Agent

The obvious solution is to remove all tracking and make the avatar disappear when someone is forced to deal with an interruption. This would be similar to how short-term absence is handled in current remote collaboration tools, like Zoom.

But this would only solve the first problem. I wanted to propose a solution that addresses all problems caused by short-term absence, so including Missing knowledge and consent, and Catching up. The concept I came up with is that of the absence agent.

A helpful metaphor for the absence agent is that of a personal assistant. When you have to leave the virtual environment, you leave behind your assistant. They can make decisions for you and take notes on relevant things. In summary:

1. The absence agent replaces the avatar of the absentee—that way, the first problem of continuous tracking is solved.
2. Collaborators can interact with the absence agent on behalf of the absent user. It acts as an interface to them. It is imaginable that the agent has access to archeologists calendar to check their availability. In my thesis, I go into further detail on what kind of interactions are imaginable.
3. The absence agent can inform the absentee about changes and progress made upon their return. This way, the delay caused by catching up is minimized.

The Outcome

Screenshots of the final prototype



press for fullscreen

I developed a collaborative MR environment with Unity and MRTK. Users could meet in a shared scene through both VR devices (HTC Vive Pro) and AR devices (Hololens 2). They can see and talk to each other, and manipulate virtual objects together.

To test the MR environment and the concept of the absence agent I conducted an experiment in which two participants collaborate-Just like the archeologists would-by sharing individual knowledge. Their task is to place 16 artifact at the right position in an excavation site.

To test the MR environment and the concept of the absence agent I conducted an experiment. In the experiment two participants collaborate by sharing individual knowledge, just like the archeologists would. Their task is to place 16 artifact at the right position in an excavation site.

I tested different versions of the absence agents based on two axises, "Similarity to Avatars" and "Interactivity". Absence agents with hight interactivity, people can record messages for the absentee.

Four versions of the absence agents



press for fullscreen

A full explanation on the design and implementation of the environment, the absence agent, and the experiment can be found in my masters thesis.

Results

The experiment provided many interesting insights about the future development of the collaborative environment for archeologists, and for the absence agent. Two of which I will present in the following paragraphs.

1. Avatars were less important than we anticipated

Participants were not looking at the other person avatar. As a result, when the absence agent replaced the avatar, it sometimes went unnoticed. When the agent looked different from the avatar, participants recognized the absence agent more. It appears that a distinct representation for the agent is more suitable. Additionally, it could mean that the agent should not appear where the avatar of the absentee is, as the users are not paying attention to it.

2. Participants were not independent enough during experimental task

Some participants just stopped working and waited - They did not want to make any mistake. Others continued but hit a point where they could not continue without the absentee. I believe this happened because the task was too linear, and a more flexible task with multiple possible outcomes would be better. This would also reflect real-life collaboration better. Additionally, having more than two collaborators could be helpful.

Conclusion

I am pretty proud of this project. To the best of my knowledge, it is the first research short-term absence in collaborative MR. Even though the experiment could not show the value of absence agents, it laid the foundation for further investigations. I believe that understanding the problems that come with short-term absence will benefit collaborative MR environments in the future.