Project Description:
Full-featured VR environment for writing, debugging, and understanding code written in an existing, real programming language. Like math, code is only the vehicle for the ideas of programming. Human beings don’t (can’t!) reason in large amounts of code, instead we rely on spatial and visual models when writing programs.
We’ll be making a physical analogue for programming that relies on practical interaction and normal human intuition to work. Imagine a table of Legos representing variables or types. Composition is just sticking two pieces together. Interfaces are real, tangible shapes. A large, complex body of code can be “zoomed out” and represented as one small, unique shape or be exploded in reverse, revealing its inner workings. Imagine those lego pieces flying around interacting with one another, showing a program running live or pausing to debug.
Its an ambitious project that I’ve put a lot of design work into– if this pitch seems vague or wishful its for the sake of clarity. I bought a rift and started writing code two weeks ago: its currently working in a very basic way.Realistically the semester goal for this project is to render code as interactible objects, allow the user to change those objects in a syntactically legal way, and execute the program. The stretch goals are to show debugging, set up abstraction layers for objects, and write the code back out to file.
The key functionality needed here is reading in code, compiling it, editing it, and writing resulting code back out. This is available off the shelf for a handful of programming languages– right now the plan is to use Python. To that end this project is going to use Unreal Engine 4 since someone was kind enough to integrate a Python runtime into the engine already. This project will not involve modeling, animation, or traditional level design. It will involve procedural geometry manipulation, animation, in-engine work, and high level language manipulation.
To the non-programmers: the most important part of this project is the design. There’s a huge amount of design work that has to be done to make this viable or interesting, so if making programming easier is interesting to you don’t worry about not bringing anything to the table.
Project Type:
Product
Interest:
Very interested in the problem on a personal level. Reading and understanding code is hard!
Device(s) to Be Used:
Oculus Rift and touch controllers. I have an Oculus Rift.
Things I Know How To Do:
Language/compilers, graphics, design.
Things I Would Need Help With:
No experience with Unreal or any other serious game engine.