CG/VR

• Dynamic Softness/Hardness Display by Independent Control of Contact Force and Area
  
  

  Various devices have been developed to present the hardness/softness of an object, but it is difficult to dynamically express the hardness because the contact force and area are not controlled independently. In this research, to express the hardness/softness of objects, the device was developed to possibly render the characteristics between contact force and area by combining the contact force with motor torque and contact area using an airbag. We performed the experiment confirming the area and force curve (A-F curve) with the proposed device compared with the theoretical model. As the result, it was shown that the device can reproduce the A-F curve of the constant Young's modulus model and dynamic changing Young's modulus model.

  
  
• Auto-calibration of Head-mounted Eye tracker Using 3D Fixation Points
  
  

  In order to use an eye tracker, explicit calibration is required to make the user look at markers in advance. Several existing automatic calibration methods have been proposed, but the method using the saliency map of the field of view image is computationally expensive, and the method using the scene model is limited to planar displays. This study proposes a novel automatic calibration method for eye gaze estimation in virtual reality that does not use saliency maps and is not limited to planar displays. The proposed method is based on the assumption that the user can move the head freely and that the point-of-regards of fixation during viewpoint movement are dense on an object surface. The method first detects fixations and then estimates calibration parameters by minimizing a cost function for the variance of the point-of-regards during fixations. The experimental results show that the proposed method can be calibrated with the same accuracy (about 1 deg) as the conventional method using the gaze information for a differentiable surface.

  
  
• Shearing Force Based on Contact Shape
  
  

  In recent years, a machine handles not only hard objects but also soft objects. However, there's a problem that shearing force cannot be accurately transmitted to soft objects with stiffness distribution due to slip. In this study, the author develops a device that applies shearing force without causing the slip by driving three end effectors with servomotors. The target is an elastic body with stiffness distribution, which models an ankle. In the experiment, it was suggested that the slip was reduced by increasing the radius of curvature from 40 to 60 mm in the stiffness distribution of the ankle. [Omori 2019]

  
  
• Gaze-driven Microscopic Vision Augmentation System
  
  

  A visual augmentation system that realizes vision beyond the limits of human eye is expected as a next-generation support tool. The gaze direction is information that represents the intention to look at the object. By estimating the gaze position and identifying the target, it is possible to extract the target of the user's attention. The author applies a visual interface that understands the user's intention to the microscopic observation. The author proposes a microscopic vision augmentation system that identifies the gaze target from the gaze direction and adjusts the focus and angle of view automatically.

  
  
• Instantaneous Cooling Model Based on Thermal Jet Engineering
  
  

  In recent years, technology for cooling objects in a non-contact manner has been developed. However, it is difficult to cool an object instantly. In this study, we propose a model that can estimate the object temperature change when cooled by the cold air flow generated with a vortex tube. An impinging jet generated by transported cold air cooled the object instantly. We construct a cooling model that can estimate the temperature changes by taking the heat transfer characteristics of the impinging jet into account. The result showed that the temperature of the silicon sheet can be predicted within an error of 0.36 K.

  
  
• Development of Presenting Hardness/Softness Device by Rendering Contact Force and Area
  
  

  For haptic presentation, many researches have been carried out, but, for hardness/softness presentation, the expression of contact physical quantities is limited and many technical issues remain. In this research, the author develop a hardness/softness presentation device that renders the contact force and area based on measuring data with independent control of DC motor and air-bag device. This device reproduce the characteristics of each physical quantity and be possible to expressing the hardness/softness of various objects.

  
  
• Magnetic Haptic Display
  
  

  In this study, the author developed a force feedback device using magnetism. A touch panel was used to display visual information and acquire the coordinates of the user's finger. A magnetic field is generated by electromagnets placed on the back of touch panel, and a force is applied to a permanent magnet attached to the user's fingertip. In addition, the electromagnet is moved according to the position of the user's finger. As a result of verification, it was suggested that the force displayed to the user can be controlled by the current flowing through the electromagnet.

  
  
• Automatic tracking air jet
  
  

  We have developed a device that automatically tracks the user's movement and injects air at any position. Air is injected to a position by rotating a nozzle that injects air using two servomotors. The angles of rotation by the two servomotors were derived from the kinematic techniques. The arbitrary position derived from the known rotation angle was input to the created system, and the rotation angle was calculated by reverse calculation. It was possible to present different forces while tracking the user's movement.

  
  
• Transmission of Tactile Sensation Using Light Communication Projector
  
  

  Interaction technology with video has been developed. However, there have been problems in multi-channel transmission of tactile sensation targeting the wide range of skin. In this research, a lot of signals corresponding to pixels are transmitted with the projector light to reproduce the tactile sensation. Signals of any frequency are transmitted by changing the brightness of the projection light.

  
  
• PID Position Control for Flying Haptic Object
  
  

  Researches related to a device that presents tactile feedback of a virtual object have been conducted. If a user feels tactile sensation without wearing a device, the available space of the tactile feedback is limited. The author proposes a haptic device that can present tactile sensation in a wide space without wearing a device by using a drone. The author implemented drone�fs flight system. PID controller was used for flight system. The author determined optimal parameters and measured step response. As a result, rising time was less than 1 sec and steady state error was less than 10 mm. [Ikemoto 2018]

  
  
• Modeling of vortex ring generation for sensation transport
  
  

  Vortex rings are drawing attention in sensory transport, and fluids can be transported far away without diffusing by using vortex rings. In this research, the author aims to experimentally clarify the relationship between vortex rings and generation conditions, and to construct a mathematical model relating both. The parameters of vortex rings were measured by using cameras, and the relationship with the generation condition was approximated by a cubic function. With this result, the vortex ring with desired parameters can be generated and the sensation can be transported far away without diffusing.

  
  
• Object animation using projection to smoke
  
  

  Space projection is a visual display technology that enables users to see 3D images in the air. the author proposes a method of space projection that enables users to observe the 3D image from any viewpoints and touch with the image by using a smoke screen that has isotropic scattering property. The smoke screen perpendicular to the projection direction is moved at a high speed, and an image for the position of the screen is projected.

  
  
• Dynamic tactile shape display for fingertip
  
  

  Haptic displays enable us to manipulate objects at a distance. However, these displays could not change their shape on users�f fingertips. In this research, the author developed tactile shape display on users�f fingertips. The device have 4 x 4 pins and presents objects�f shape by controlling pins�f displacement independently. The author miniaturized the device by means of ultrasonic linear motor and displacement sensor without interference. [Yamada 2018]

  
  
• Thermal AR system
  
  

  AR is an information enhancement achieved by superimposing virtual senses on real objects. Since the thermal-display is placed between the object and the skin, it was thought that thermal AR was difficult. To solve this, we use thermal phantom sensation to display heat without touching the finger-belly. Thermal phantom sensation is an illusion phenomenon which gives thermal perception between two heat stimuli, and the sensed position is biased to the strong stimulation. We experimented whether it could be used as a thermal-display by changing the sensed position. As a result, thermal-display was possible in the proximal-distal direction of the index-finger.

  
  
• Three-dimensional cooperation stage prompter
  
  

  With the development in the field of computer graphics in late years,opportunities to communicating in virtual environment increase.But 3D user interface has some limits such as complicated operation. The finger gesture attracts attention as easy input. In this study, depth camera recognize the joint positions of the finger and recognize a gesture from relative position of recognized joint of the finger. we aimed at building simple and easy interface which doesn't need skill by operating a 3D object in a virtual space depending on a recognized gesture. In addition, I tested it and confirmed that intention transmission on the virtual environment was possible. [Fukuda 2017]

  
  
• Wind Drag Navigation
  
  

  This research proposes a haptic device for navigation. The author used the drag caused by the wind force in order to make a portable device. The direction that the user feels is changed by rotating a fan. In the experiment, the author confirmed that the device is possible to feel the left and right direction. [Haraguchi 2015]

  
  
• Virtual Highline
  
  

  "Virtual Highline" enables a user to walk on a rope at heights safely by using a HMD and a foot haptic device. The user wears HMD and walks on the foot haptic device. The HMD presents landscapes as if the user see on the rope, and the stage wobbles like a real rope. The user can walk while staying in the same place by using a belt conveyor stage. The user enjoys thrill walking on the unstable rope at heights.

  
  
• Oil Bubble Display
  
  

  Oil Bubble Display is the novel flexible display which consists of the floating oil bubble on the water. User can interact with the image projected on the oil bubbles by deforming and fusing and separating them. According to mixing the oil and the material which scatters the light, the shape and position of the oil bubbles can be measured, and the image can be projected on them. User can interact with the color and shape of the projected image by operating the oil bubbles.

  
  
• Virtualization of Liquid Based on Waveform Observation
  
  

  Smoothed Particle Hydrodynamics (SPH)-based method is used to simulate the highly deformed fluid, and Computer Graphics (CGs) in movies or games are often created with SPH-based method. The physical constants are not suited for the parameters of SPH, and the parameters setting is a hard work. In this research, we propose the method to virtualize the liquid by comparing the waveforms between the real liquid and some kinds of simulated liquid with different parameters. Using this method, we can get the parameters describing the viscosity and the rest density, and virtualize water and salad oil. [Takemura 2015]

  
  
• RGB-D Information Processing for Framing
  
  

  This is a method for a high-visibility projection onto a proper surface in the variable environment from a mobile object. In the proposed method, a proper surface for the projection is chosen based on a photometric and geometric property of the surfaces in the scene. For the acquisition of those information, the RGB-D camera are used. Then, the image is projected onto the surface chosen by proposed method through the relative position estimation between the projector and the camera. [Danbara 2015]

  
  
• Haptylus: Stylus for Unified Manipulation
  
  

  This study proposes a haptic device using a retractable stylus and a tablet PC. A vibrator and DC motor are attached to the retractable stylus. The user can perceive a texture of the virtual object by vibration and force by shrinkage of stylus by DC motor. The proposed system gives a sensation like that the stylus is inserted into the tablet PC when the user takes the retractable stylus and pushes toward the tablet PC. In addition, the system estimates a position of the user's viewpoint and projects the image according to the viewpoint. [Nagasaka 2015]

  
  
• Puppet-Tone
  
  

  Recently, development of a system that is able to improve the singing ability is desired by the spreading of karaoke. When we sing, we are singing to suit the environment sound, such as ambient sound and accompaniment. In this study, by presenting the accompaniment sound corresponding to the deviation of the pitch of the singer, we aim to develop a system for manipulating the pitch of the singer and improve singing ability.

  
  
• Virtual Hand Sculpture
  
  

  This thesis describes a modeling system for designing with a virtual three-dimensional model with embodiment input. Embodiment input means that the direct output is performed for the input of the user. In this system, user observe the object by movement of the head and model a virtual object by hand.

  
  
• Haptic AR of Using String for Training of Catheter Procedure
  
  

  In this study, the author uses the real object for blood vessel, and simulates virtual vessel in the computer for a lesioned part in head. By integrating the contact force between the blood vessel and guide wires from the real object and the collision force between virtual guide wires and virtual blood vessels, the total contact force is applied to a user with a developed haptic device. In this way, the device allows to produce reaction force. The guide wire is moved linearly along the cylindrical tube. The haptic device has two degrees of freedom with the motors and strings.

  
  
• Operation Guidance System with Work Steps Superimposition
  
  

  In this study, the author proposes the system that monitors the statements of target objects by attached sensors, superimposes the stage of operations and operation parts on HMD, and warns the operator for a mistake. This time, the author made the box that imitates a printer with the accelerometer to detect the inclination of the lid and the reed switches to detect paper kind and direction. [Wakita 2014]

  
  
• Tactile Augmentation for Tool Manipulation
  
  

  In the handheld tool manipulation with index finger and thumb, tactile sensation at the digital pulps influences the perception and recognitions of the physicality of a object and the state of the manipulation. In the tool manipulation, including surgery, tactile augmentation at the digital pulps to notify the state of the manipulation seems to enhance the efficiency and the accuracy of the manipulation. This study aims at developing the smart tactile display with high usability which can be applied for the delicate manipulation and applying to the actual manipulation. In the previous works of the tactile display, tactually perceived position corresponds to the stimulated position. Therefore, the manipulation will be disturbed by the stimulator. On the other hand, this study proposes Spatially Transparent Electrotactile "STEleT" display which can separate them. Especially, we focused on the anatomical nerve structure and the conductivity of the human body, and obtained the sufficient usability and the function of tactile representation by using electrical stimulation at the middle area of a finger. The developed tactile display was utilized for the notification of the pre-determined danger to avoid the invasion during the tracing task with a pen. It was confirmed that user could avoid to invade the dangerous area in spite of the no visual effect. [Yoshimoto 2010]

  
  
• Haptization of Multifunctional Dental Image
  
  

  The integrated and interactive image of the multimodal images such as X-ray CT image and MRI image is called multifunctional image. The effective display technology is required for the multifunctional image, especially dental multifunctional image. This study aims at developing a pointing and display system which haptizes the anatomical information of the multifunctional dental image by using tactile display. In order to haptize, a three dimensional dental shape data, a panoramic image of the tooth and a facial shape data are registered into the same virtual space, and the voxel data including anatomical information is generated. Spatially transparent electrotactile (STEleT) display was selected to present tactile information for its usability. User can point the virtual space with a manipulator and feel tactile signal according to the anatomical information at their digital pulps. The developed system is expected to became a new effective tool for the diagnostics and the education in dentistry .

  
  
• 2D-3D Registration of Multimodal Dental Images
  
  

  These days, acquisition of 3-D volume data becomes popular in medical field. However, to obtain 3-D data in dental field is still difficult.In dental diagnosis and treatment, dentists use dental cast models which is obtained from dental impressions.Dentists also use 2-D panoramic X-ray images, because dental cast models don't have information inside alveolar ridge.The authors propose the registration method between 2-D panoramic X-ray image and 3-D volume data. Since to obtain the position of dental arch from 3-D volume data is difficult, the proposed method achieves 2-D and 3-D registration in following way. First, a dental arch is extracted from an oral image and is projected onto the 3-D shape of the dental cast model. Second, a 2-D panoramic X-ray image is segmented into individual tooth images and these tooth images are arranged in 3-D shape of the dental cast model based on the projected dental arch. [Shigeeda 2011]

  
  
• Tactile Touch Panel by the Displacement Control of Shape Memory Alloys
  
  

  When touching an object, tactile sensations are felt, such as surface coarseness, unevenness of the object, pressure, and the transformation of the skin. In Virtual Reality, electric and mechanical displays have been proposed to produce tactile sensations. However, the displays have never produced sensation of skin transformation and vibration at the same time in mobile displays. This study aimed at presenting tactile sensations by using a shape memory alloy enabling skin transformation and vibration at the same time. In this study, a shape memory alloy is used for the tactile display. The shape-memory alloy�fs displacement is generated by PWM controlling and vibrate by frequency control. A shape memory alloy wire was connected to a slider and adding the tension to the wire.The tactile sensations were able to present on a slider. Vibrations could be changed corresponding to the frequency and pulse width in PWM control.

  
  
• Visual and Haptic Touch Display by Tension Control of Flexible Seat
  
  

  We propose a haptic touch panel, which displays unified visual and haptic sensation of pushing an elastic object by finger. In nature, while a user touches and pushes an elastic object, contact area between a fingertip and the object changes as well as reflection forces. The proposed display presented stiffness by controlling both contact area and force. The contact area was controlled by changing tension of transparent elastic rubber upon the panel. The result of experiments showed that the developed system enabled to change contact area by controlling tension and to present different stiffed objects. [Shigeta 2010]

  
  
• Tactile Rendering by Using Electrical Stimulus Pattern
  
  

  We feel tactile sensation by touching a object. Manipulation with a user interface such as a mouse seems to be equivalent to the touching a virtual object with a virtual finger. Tactile feedback according to the manipulation of the interface may enhance the efficiency of the recognition or accuracy of the manipulation. This study focuses on the development of the small tactile display which can present texture sensation, and proposes tactile rendering method by using electrical stimulus. Especially, texture representation by using a uni-anodic stimulus will be achieved by characterization of the skin displacement and its reflection in the stimulus pulse rate. As an example of the texture representation, roughness was introduced. User could discriminate the texture in several stage of the roughness according to the shape of the object surface. [Yoshimoto 2009]

  
  
• Haptic Display System by Using Dilatant Fluid
  
  

  Dilatant fluid which is made from the water and starch changes the state according to the applied force, and provides the haptic entertainment. Therefore, a new interactive haptic system in which users can generate and blend haptic sensation with their hand is expected to be some clues for the explanation in an intrinsic property of the haptic sensation. This study aims at establishing a new haptic display technology by controlling the dilatant fluid to present multimodal haptic sensation. Especially, we proposed a haptic glove which has filters of the starch and sucking/ejecting tubes to control the density of the dilatant fluid. Users feel haptic sensation according to the controlled state of the dilatant fluid. The experimental demonstration using the proposed haptic device indicated that the device can present "stickiness", "hardness" and "roughness" sensations. Furthermore, it was suggested that the proposed device can control the three physical quantity corresponding to the shape, viscosity and stiffness of the material from the factor analysis of the answered sensations.

  
  

• 360-Degree Observable Fog Display
  
  

  Spatial image display is one of the image display technologies. Spatial image display means a image system that users perceive no screen or subtle screen and the image in real space. Spatial image display can present stereoscopic effect although it displays 2D image because, for example, real image display has no frame. The study proposes the system where a cylindrical fog screen enables users to observe the image corresponding to the view point. The system projects different images from three positions on the cylindrical fog screen that is generated by ultra- sonic humidifier and circular nozzle. Since light is scattered mainly forward by fog the projected image can be observed only by users on the opposite side of the fog. Therefore users observe one image although multiple images are projected on the fog display. Author confirms that by moving observing points, users can observe stereoscopic image because of motion parallax. The experimental results show that Projected images are corrected depending on projector�fs position and axis direction. [Yagi 2011]