13 Jul

Abstraction of Gesture Detection and VR-Based Design

Cheuk Yin Phipson (Phipson) Lee

Earlier this week, I was fortunate enough to discuss and share my learning across several meetings with teammates in my lab, my supervisors, and also fellow researchers who were also immersed in their individual studies. Initially, my main goal and motivation for pursuing this research project in furniture design was to investigate ways by which furniture designers could employ natural, intuitive 'sketching gestures' to generate a furniture object with measurable specifications in 3-dimensional space. In the same way artists in the past sketched with pencils and pens, studies, documented in my project proposal, have indicated the benefits of using physical interactions as a method of designing art in 3D space. Further details can be found in my project proposal, but as a short summary:

  1. The traditional 2D Graphic User Interface limits designers' abilities to translate their ideas into the physical, 3 dimensional environment.
  2. Contrary to the old, outmoded method of 'sketching on paper', which is easy to learn, offers great flexibility, and, as evidenced in multiple user experiments, enhances immersion, satisfaction and freedom when designing.
  3. Many designers typically lack real-time collaborative utilities to cooperate and design with peers in the same environment. In addition, virtual designs are often alien to the physical environment in which the design is to be implemented.

An initial model of the proposed product was designed 2 weeks ago, based on the literature, deliverables, and goals outlined in the project proposal.

Here, home state is treated as the starting point of the application, which immediately translates into a sketch state- utilizing the camera that displays the virtual environment, users are introduced into an open world where they have access to sketching mechanisms that are based on existing libraries such as Tiltbrush. Using this sketching state, users can not only select existing objects, but also sketch new ones and adjust its parameters. They can also interchange with collaborative channels and multiuser services that can be access through Unity's Photon server.

However, over the course of this week, after learning more about the available resources and features of the backend furniture compiler, I realized that while the premise of this design concept offerred an unprecedented and intuitive method of designing furniture (in contrast to industry-standard CAD devices that utilize a mouse-and keyboard setup to develop 3D products), the use of 'sketching' as a method of 'parameterization' for the existing backend compiler is complex and fails to utilize the pre-existing self-generative methods that the compiler users to create manufacturable designs. As a result, while I had initially focused my literature research on sketching mechanisms and translating sketches into virtual, manufacturable objects, to employ the existing utilities of the compiler, I would put my work on sketching aside, and focus primarily on manipulating existing virtual furnitures designed by the compiler.

After initial discussion with my research partner, June, we had decided on the following specifications:

Upon further review from peer researchers in our most recent lab meeting, I had an enthusiastic discussion with Tsang-Kai regarding the methods by which this research will easily help users learn how to design furniture. In particular, he questioned our metrics and assumptions that virtual reality and augmented reality are easy-to-use. Further evaluating the feedback from this duscssion, and exploring the research that I had compiled in my proposal, I realized that though there have been studies that demonstrate the qualitative benefits of this project, they have not offerred reliable inventories or metrics for quantifying the learning curve required for industry-standard CAD software. This stood out to me as a potential loophole in subsequent parts of this research, as I am looking to produce several user studies measuring the benefits and usability of our research.

In the coming weeks, while concomitantly developing the draft product, I will be further investigating apposite literature relevant to specific virtual manipulation of geometries in our aforementioned model. In particular, I will also research methods and metrics of quantifying the usability and ease of learning of our research, in contrast to industry-standard devices. The models and abstractions that have been displayed here will continue to be developed.

Next Post Previous Post

Search

Popular Tags

foray origami robots blimp ducky vr edg SLAM computer vision computational design roco slam Origami rocolib arduino RoCo design Robot Compiler simulation Motion Planning autonomy edg-model mesh reinforcement learning; multi-agent; mean-field; logics create; kalman filter; motors imu friction mesh network tank esp32 networking contacts multi-agent pushing auxetic machine learning dynamics esp now visual processing wireless control communication webapp pcbs structure ROCO i2c path planning cars Robotica reinforcement learning computer skill openmv test motor dataset visual DARPA odometry Battery surface structure power consumption dash port-arrays modeling origami paper frequency diversity semantic infrastructure auxetics feedback gondola machine vision Consensus bistable auxetic controller gestural intuition mixed reality unit cell robot crawler gestural interface open house network create; kalman filter; EM; refactoring 4-wheeled darpa; create; icra2021 apriltags lidar logic control FORAY perception Quantum Information crazyflie 433mhz optimization propeller Foray structures sensors environmental design radio DER Kullback-Leibler Divergence lasercutter tutorial von Mises Fisher Distribution IR simulink legs project management localization distributed optimization blynk Auxetics painlessmesh experiment results offline reinforcement learning exam research proposal altimeter thrust PCB sensor fusion gate or webots multi-agent path planning UAV RL integration test arnhold joystick balloon test vision processing pushing problem modeling contacts mesh networking motor configuration nodemcu swarm object detection fusionbot ros2 rviz2 gazebo conference matlab arduino nano decentralized surfaces keyboard IROS Integrated Demo throughput peripheral device paper review reinforcement learning; motorized boats Writing slam; paper review rhino Short Post range distance surp origami robot nylon pvdf elastic polymers plastic hot plates reinforcement learning; multi-agent; mean-field; game theory; mean field; reinforcement learning current reinforcement learning; multi-agent; Xbee Zigbee robot compiler spi contact neural network dynamic modeling quasi-static modeling edg-examples Kalman Filter port-array von Mises Fisher von Mises-Fisher literature review multipack car Zero Elastic Energy modes swarm control solar cars system ID serial monitor edg-ide Discrete Elastic Rods ceres debug Consensus Algorithms coriolis force dynamic model proposal wtf deep learning pdms interpolation wireless signal strength reinforcement learning; multi-agent; decentralized misc reinforcement learning; multi-agent; mean-field; stationary demo video flytrap papers steering structured field pivoting differential power detectionrate color shape edg-refactoring pain python vision Machine Learning Q-Learning Model-Free release conferences quasi-static pushing liquid crystal elastomer nonholonomic systems iot grant feedback-controlled controls trajectory optimization surface contact Characterization tool ML multi-agent path finding Liquid Crystal Elastomers Learning kinematic_model reservoir computing 2.4 GHz concept 433 MHz Bayesian Network creative Computer skill internal silicone integrated system instance segmentation marginal cost pricing bootstrapping Bayesian networks ensemble methods offline Rl hardware manipulation COVID 19 paper cutter ESP32 kalman filter; multi-robot state space delta tolling buoyancy chiral square unit cell pressure distribution model prediction thrust configuration grid fabrication firmware protobuf #activecapturingmechanism #claw 3D-CHESS; kalman filter boxboat tracking model-based RL von Mises Distribution slam; flowchart; spring-mass 3D-CHESS; 433 rssi signal strength multirobot system writing in-situ design buoyancy equations boat deployment cell cameras der newton's method Barycenter Euler-Bernoulli beam sma-based actuator game theory Bessel functions von Mises-Fisher Distributions ide HDL board design battery current sensor ZMP center of friction Multi-Agent Systems code balloon semantic segmentation MIMO communications ICRA ICML guassian processes probabilistic neural networks graph generators MBRL batch learning ros edg-compiler multirobot optitrack adaptive control motor driver submission DNN kalman filtering point cloud 2.4ghz latency scenic codesign presentations machine training preliminary exam multiple devices inkscape material multi-object tracking ICLR Python actuation catching mechanism integration flapping color_segmentation lab_data result segmentation on multiple object and centers finding competition prelab multiple blimps pid defender Synthesizable Radio 3d printing kalman filter motor distribution barometer detection lift force depth estimation cad solidworks design tool pixracer pixhawk ardupilot flight controller regression fly test sensor selection