High Speed 3D Modeling – First Steps

Hello Bloggers!

This is Andrew Chase and Bryce Sprecher, and we’ve started working in the LEL this semester under the direction of Kevin.  We haven’t make any posts yet because most of our efforts have been focused on deciding exactly what we want to do, and how we are going to approach it.  Now that we’ve had a chance to start, here’s our plan:

High-Speed 3D Modeling Through Agisoft

Agisoft Photoscan is a software program that allows the reconstruction of 3D models through digital images.  We plan on starting with creating simple models of objects, and progressively moving onto taking high-speed videos of some phenomena (fluid dynamics, popping of popcorn, combustion of materials, etc) and constructing models of this process.  From there, we can play back the event in 3D (first by conventional means of 2D display, and hopefully Oculus/CAVE integration) in slow motion to relive the event up close and with detail.

We obtained a Nikon J4 Digital Camera with two different lenses (10-30mm, and 30-100mm), which is capable of capturing:

  • 1080p at 60 frames per second (fps)
  • 720p at 120 fps
  • 768×288 at 400 fps
  • 416×144 at 1200 fps

We’ve tested several different phenomena at different resolutions, and found that the 1200 fps capture would work great at obtaining high-speed imaging of solid objects, without much need for texturing detail.  If detail was needed, the 400 fps still captures high-speed action at very acceptable detail.  Here are some examples:

Dice Roll @ 400 fps

Egg Crack @ 400 fps

(NOTE: I’ve tried for a while to embed the videos, but it doesn’t seem to work on this blog)

With some post-processing, we believe we can further enhance the high-speed effect (reducing playback speed further, adding motion blur/fake frames, etc).  In addition, these were shot in low light environments, and it is apparent that a lot of light will be needed for this kind of capture (High fps = Shutter open for short time on camera = Less photons hitting sensor).

Our next steps involve trying to make some models and see how Agisoft Photoscan handles video input (whether it will handle a group of frames at once as a video, or will we need to do it manually, frame-by-frame).  If everything looks promising, we can look into getting more cameras in order to capture different perspectives of the phenomena to create a model.

chat – A lightweight tool for interacting with others in virtual reality.

Chat allows users in a virtual reality (VR) homespace a quick, easy, and fun way to send voice messages, stickers, and even presents to other VR users.

Rather than copying traditional messaging interfaces into a VR homespace, I will explore and execute upon methods that have emphasis on light and natural forms of communication.

The following examples show mockups for a four different types of communication that could work in VR:

  • Voice – Allows a user to capture a voice message and quickly send it to another user.
  • Text – Allows a user to either choose from predefined responses or use voice-to-text.
  • Emoji – Allows a user to select from pre-defined emojis or stickers in a quick way.
  • Present – Allows a user to send a virtual gift to instantiate in the other user’s scene.

chat Concepts-01

Mockup showing voice, including a microphone button that will allow them to record their voice.

 

chat Concepts-02

Mockup showing text, including a predefined message.

 

chat Concepts-03

Mockup showing emoji, allowing users to pick from a grid of reactions.

 

chat Concepts-04

Mockup showing present, including a 3D model of a dog to be sent to another user’s space.


 

Talking Points

  • Traditional 2D interactions versus rich 3D experiences
  • Implement a system of basic chat and simple 2D games across players
  • Explore different methodologies
  • Scaling from 2 people in a chat to a whole group of people and what that entails
  • How to share information and resources
  • How can people collaborate differently in VR compared to in what we have today?
  • Potentially measure the effectiveness of simple chat experiences vs. full 3D experiences
  • Does a “2D” social experience feel natural in VR?

Existing VR social spaces: AltspaceVR, VRChat, Convrge, vTime, Oculus Social (Alpha)

Deliverable: A collaborative social application (in Unity) (“2D”) that can run with two or more players and a short study on how people respond to the application and what steps can be taken to improve the application.