I feel like although VR in the perfect mindset would be used to achieve great things through education and training as well as advertisement. But what is seen to be the most common thing that ties people to one another is social media. The tool of VR can be applied to so many things but when applied to the way humans communicate, it would provide an new platform which might be dangerous in the way it would immerse individuals into complete different realities thus escaping their own.
There is this show that I watched previously called “Kiss Me First”, where you would follow the rise of technological advances where they managed the sense of touch in VR. The show represents quite a dystopia, but i feel like it’s an interesting way of thinking where VR is heading. It isn’t just a social media platform but a game as well, and that is sort of a new idea to cross these two things.
One of the most effective representation in VR is letting interaction that is not possible in real life happen. Learning a language can be difficult. We find it hard to practice it, especially due to the fact we usually don’t have a person around us speaking the language we are trying to learn. However, one of the most effective ways to learn language is to talk to other people.
There is a VR application that lets you learn languages through VR experience. This will let people have the interaction that we lack when we learn new languages through books. Because lots of people find learning a new language difficult, I believe that representation of VR in this area would be very helpful.
A form of representation I feel would be suited for VR medium for doctors during a surgery.
It creates an interface to the world where the doctor can comprehensively assess the patient’s physical situation, instead of just seeing from a single dimension screen and being influenced by the surrounding environment. It is common nowadays that a doctor’s hands are operating a surgery while turning his/her head to another direction to see the instant image on the screen, this is an “inhumane” design for doctors because it loses the coherency of eyes and hands. What’s more, there are multiple other external conditions might influence a doctor’s judgement – the light in the operation room, the anxious patient, or even the breathe of the surgery assistants. Therefore by implementing VR into this specific situation will minimize the external influence so the doctor could have a better performance.
By operating with the precise equipment attaching to VR, the doctor will be able to focus on minor details in the surgery (eg. sewing the wound) and eliminate the failure of false judgment.
In molecular and cellular biology there are two three dimensional aspects that are often portrayed in two dimensions, these are molecules and their interaction within pathways. Molecules are usually drawn as simple blobs or shapes in order to better visualize the different domains that have specific functions (proteins have very complex shapes). With these simple shapes, their interactions are usually connected with a vast map of arrows and inhibitions which form the pathways that biologists study and use to develop function specific drugs or site specific research. This decision to go simple and 2D has made studying and using biology a lot simpler but it has taken out a very important factor, molecules and pathways have movement. This movement, which is important in the interaction of proteins, depends vastly on size and surroundings which are all three dimensional. Naturally, to model this mathematically is quite complex and requires a lot of prior knowledge and computational power but once achieved has great benefits. If these pathways could be brought into a dynamically moving intractable three dimensional world, then there would be the possibility of better research and understanding into medicine. By better understanding what happens when certain interactions are “physically” and “visually” removed there would be less wasted effort in pathway structure and experimental design.
My perfect representation of a VR that will be suited a for the medium is a social media network. I know that sounds a bit strange and introvertish but that is how I see VR. We use the social media every day. Facebook, Instagram, Snapchat and etc. Why won’t we use them in VR. Imagine you are a person that walks in the hall of your friends, watches their pictures, hit the like button, but you are actually doing it in a VR world. I think that would be an amazing representation of VR.
When
designing a building, often, no one knows what the building would really feel
like to be inside until it is built. Now,
with VR, we can explore the inside of a building that hasn’t been built yet, or
even a building that is impossible to build.
The form of representation I would like to write about in this post is based on my own experience in creating three dimensional models. I went to architecture school for a year. We were encouraged to start thinking about our projects by making sketches. Though I would start out this way, scribbling in a sketchbook, I would quickly become frustrated by how, though my sketches managed to capture the feel and aesthetic of what I wanted they never managed to convey a sense of the space. I quickly moved on to making rough three dimensional sketches or sculptures with bits of paper, pizza boxes and a box cutter. This really helped me think spatially, to see in one go what it would take a plan, several sections and an isometric view to see before was immediately visible with the three dimensional sketches.
A few years later, I was teaching myself Maya. The modeling capability of the software was more powerful than my limited real life model building ability. Yet the interface of the 2D surface of the screen was a barrier to being really able to see what I was doing as I built. I would move a vertex a certain amount and once I rotated the camera, I would realize that I had moved it too much in the x, y or z direction without even noticing.
VR would be
useful in taking a tour through the model of a building to a client. But it
could also be extremely valuable at the sketching, conceptualizing and designing
stages as well. The software would consist of ‘dynamic material’ that the user
can manipulate by holding onto and dragging and scaling surfaces, vertexes,
edges and volumes, like on maya.
The
software would also have two modes, a miniature one where the architect can
tinker with the model and change things and an immersive mode. Thinking about
the scale of the body is also extremely important in designing architecture. The
architect would be able to move from a miniature scale to placing themselves
inside the model as they work on it.
There are some elements of a building that VR would not be able to capture as yet, like heat and air currents but VR would be excellent for representing and creating a sense of space.
During Bret Victor’s talk, I loved learning about William Playfair and how he invented the bar chart and other graphical methods to represent data. Related to these methods are “explorable explanations,” abstract representations that show how a system works or a way for authors to see what they are authoring without the black box of code.
Data visualizations are a powerful representation that is suited for VR. Though there are some visualizations that have been developed in VR, they usually rely on the game engine to navigate between charts or they will have some irrelevant motion like the bars rising in a bar graph when it is first loaded. I think with VR we can do more to incorporate the different modes of understanding that Bret Victor mentioned. For instance, we can build upon our spatial understanding to understand quantities, time, associations between nodes of information, or even how the charts are organized (like a library of books). We can build upon our aural understanding through having audio explaining the data and walking the user through it at a level specific to the user’s experience.
VR can make an data visualization an interface to information that makes the data accessible and easy to understand through abstraction. However, there is also potential for it to unpack the layers of abstraction and show how the data visualization has been made or even give the context behind the data. For instance, if there is a chart showing the amount of snowfall, could the user be immersed in the environment showing the snowfall and the data visualization of its levels? Data visualizations are a person’s stories of that data, so they are already created in mind with a specific objective for their audience. The trouble with these visualizations is that they tend to dehumanize the context behind that data, so VR really has the ability to use its potential for immersion to help the audience better understand the story. However, it is important for VR to not exploit this potential and to falsify the data through creating a specific immersive experience that causes a different perception of that data. I also think using VR to visualize data relates to the dynamic models that Bret Victor discusses at the end of his talk. Imagine data being updated in real time and seeing how the representation changes: the bar increasing, a point on a line graph being added, etc.
I think one of the biggest strengths of VR is the fact that the player feels immersed in an alternative environment and feel an intimate connection to that world and objects within it.
Given this feature, I think this medium is fitting to represent building of relationships between people – a simulation of how people interact, connect, and bond.
multinational friends in VR talking to the player
I first thought about this idea because of the environment we’re in at NYU Abu Dhabi, where we have students and faculty coming from literally everywhere around the world. I was just serving as a Peer Ambassador in the last Candidate Weekend where I talked to prospective students and realized how it is sometimes overwhelming for people who have never travelled abroad before. I think simulating human interactions through VR could help people like these students to know how to situate their thoughts and conversations in a global context.
In this sense, VR could be used as an educational tool for people wanting to learn about people hailing from all over the world. For this specific representation, it could incorporate intricate details specific to different cultures and societies by simulating the different reactions and responses from characters within the VR world coming from different countries. The player can meet and converse with these characters and be informed about the nuances of different cultures.
Such representation in VR provides an interface for interaction where people learn to approach people of different backgrounds, ask informed questions, and know what factors to keep in mind when conversing.
But this leads to questions such as
Why not just interact with real people?
I think such representation of multi-cultural and multi-ethnic in VR is definitely in no may suggesting that this is the only and most accurate manifestation of what these interactions would look like, but rather is a way for people to get started thinking about how to act, approach and behave when they are in such situations. There are benefits to meeting people like these characters in person, but for those who fear making mistake and want to avoid being overwhelmed, this is a good option to try out.
What does VR bring to this specific form of representation?
I think having realistic characters to guide you through human interactions can prove to be very important for people not just wanting to “practice” and get to know other cultures before meeting people in person, but also for those who actually struggle to socialize and are introverted – VR can strive to provide an experience as similar to reality as possible.
As Bret Victor mentioned in his talk, The Humane Representation of Thought, “the powerful medium is what powerful representations to be spread,” the medium is what determines how and to what extent the representation can be executed to its full potential.
I think that the medium of virtual reality allows the user to get indulged in an environment that the user cannot or is difficult to get exposed to. For example, the medium of virtual reality can allow an ordinary person to feel as if he/she is walking on the surface of the moon, which only skilled and selected astronauts can do. In fact, virtual reality allows the user to be an active participant rather than a passive observant. What I mean by this is that with the use of the hand controllers, the user has the ability to maneuver what is inside the virtual reality. From this essence of virtual reality, I think that a form of representation that I feel would be suited for the medium of virtual reality would be “thinking,” more specifically creating “dynamic environments-to-think-in.”
Visual Image of Bret Victor’s Dynamic Environments-To-Think-In
In a sense, the medium of virtual reality “treat[s] the human beings as sacred” (Victor). Virtual reality can give superpowers to the user by giving the user agency to see and alter the virtual world they are in. Let’s say that the user is placed in a lone island and the user can see the setting through the headset. This user, if given the option, can perhaps light a fire or cook food, using the controllers at the deserted island. More realistically speaking, the user can craft his own image of his house in his virtual space, which he can use it to execute his thoughts, and also come up with new ideas. Such interface would trigger different parts of the users’ brain and allow the user to think in a different manner. Because thoughts and ideas tend to arise when one changes his environment, the medium of virtual reality would be effective in providing that alternate space.
In “The Humane Representation of Thought,” Bret Victor discusses different modes of understanding mediums, specifically the use of different sensory channels and enactive, iconic, and symbolic methods. At the moment, virtual reality is most typically represented using different gaming systems, such as the Vive, which allow virtual reality to be relatively dynamic. The user navigates and understands the game through action, image, and language-based representations. These games also appeal to various sensory channels; they are visual because of the graphics, they are aural because of sound effects, they are tactile because of the use of controllers, and they are spatial because of the 360 degree aspect, complete with moveable depth.
What is perhaps lacking in this current representation, however, is how the user can understand the virtual reality medium kinesthetically. There are not very many games out there, to my knowledge, that involve movement of the user’s body in a way that matches actual reality effectively. The bow-and-arrow activity in the Unity example world, for instance, does not accurately match how you would shoot an arrow in real life. It takes into account aim, a small pull-back motion, and the push of a button in order to shoot the arrow. When shooting an actual bow, there are several other aspects that go into how the archer’s arrow will shoot. The position of the archer’s elbow, for example, is very important. There is also tension in the bow’s strings that you are not able to feel in your fingers and arms with Vive controllers.
A well-suited representation for virtual reality, therefore, would be one that better takes into account the kinesthetic mode of understanding without compromising other modes of understanding the medium. Sensors all around the body to better map body movements, for example, could be a possibility.
