Channel Title: Ishikawa Group Laboratory Channel Description: Ishikawa Group Laboratory is researching on 1) Sensor Fusion: high speed robot and visual feedback, 2) Dynamic Vision System: 3D image tracking and adaptive optics, 3) System Vision Design: high speed vision and applications, and 4) Active Perception: interactive human interface and media control. Research Institute for Science and Technology, Tokyo University of Science Second YouTube Channel of our Lab.: https://www.youtube.com/IshikawaLab2 Former name: Ishikawa Senoo Lab., Ishikawa Watanabe Lab., Ishikawa Oku Lab., and Ishikawa Komuro Lab. Japanese name: 石川グループ研究室 <video> A sealant dispensing robot for applications with moving targets Description: Towards the goal of realizing next-generation smart manufacturing, we developed a sealant dispensing robot to deal with moving targets of no prior knowledge (shape, placed pose and position, moving speed, etc.) based on our previously proposed dynamic compensation framework. To realize accurate sealant dispensing for works of unknown shape that are randomly placed on a moving conveyor, the robot was designed based on a coarse-to-fine strategy such that an arm part (2-DOF) for long-term adaptation of uncertainties in a global manner (global planning), and a hand part (2-DOF) for real-time adaptation of uncertainties in a local manner (real-time error absorption). Specifically, the hand part was utilized with 1,000 fps visual feedback control. The developed robot can be either work in the auto mode without any human intervention, or work in a human-robot mode to collaborate with human. http://ishikawa-vision.org/fusion/DispensRobot/index-e.html </video> <video> VarioLight 2: Wide Dynamic Projection Mapping in Rhythmic Gymnastics Description: We have realized the combination of wide dynamic projection mapping and rhythmic gymnastics performance. "VarioLight 2" system can realize dynamic projection mapping onto a moving ball surface without any projection misalignment. Sticky and high-resolution projection is possible even for dynamic, irregular, and wide ball motion. We can observe clear and contrasted projected content, and visually grasp the ball rotation. Lively rhythmic gymnastics and wide dynamic projection mapping create a beautiful visual harmony. This technology will lead to further live applications such as media art, entertainment, and sports training. VarioLight 2 web page http://ishikawa-vision.org/mvf/VarioLight2/ VarioLight 2: Dynamic Projection Mapping for Ball Sports https://www.youtube.com/watch?v=2eyNXwjY0R0 VarioLight: Dynamic Projection Mapping for a Wide Range Performance http://ishikawa-vision.org/mvf/VarioLight/ https://www.youtube.com/watch?v=XEseo-orRDI </video> <video> Bilateral Motion Display : Multiple Visual Perception Using Afterimage Effects for Specific Motion Description: "Bilateral Motion Display" produces user-oriented visual perception by using afterimage effects for specific motion. Initially, it seems that the displayed patterns do not reveal any information; however, seen by a user moving his/her gaze in a certain direction and speed, they are spatially integrated to appear as 2D afterimages. The system expands the range of display expression and has various potential applications such as in road signs. * Abstract: http://ishikawa-vision.org/perception/bilateral/index-e.html * Details: https://dl.acm.org/doi/10.1145/3359996.3364241 H. Ikeda, T. Hayakawa, and M. Ishikawa: Bilateral Motion Display: Strategy to Provide Multiple Visual Perception Using Afterimage Effects for Specific Motion, The 25th ACM Symposium on Virtual Reality Software and Technology (VRST2019) / Proceedings, Article No. 17. </video> <video> High-speed color projector toward the world of 1,000fps vision Description: 1,000fps sensing, image processing, and projection enable real-time overwriting of objects. We, humans, will know the world of 1,000fps vision and how slow our eyes are. Web page http://ishikawa-vision.org/vision/hscp/ BGM: Rhythm and Booze by Twin Musicom is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/) Artist: http://www.twinmusicom.org/ </video> <video> 1 Millisecond Vision Pioneers a New Era Description: Our laboratory in the University of Tokyo moved from Graduate School of Information Science and Technology (IST) to Information Technology Center (ITC) in April 2020. This video shows a history and recent progress of our high-speed vision research until March 2020 as well as basic concept and active attitude of our research. Our affiliation after April 2020 is Ishikawa Group Laboratory (Project Professor Masatoshi Ishikawa), Information Technology Center, the University of Tokyo. We are conducting research on high-speed vision and its wide range of application systems without a break. URL: http://ishikawa-vision.org/ </video> <video> Accurate and Robust Inter-vehicle Distance Estimation with Stereo High-speed Vision Description: We propose an accurate and robust inter-vehicle distance estimation method using high-speed stereo vision. The framework involves two driving mechanisms. The first mechanism performs accurate and stable tracking with an algorithm optimized for stereo high-speed vision even under intensive vibration. The second mechanism estimates inter-vehicle distance via a highly accurate scale estimation and aggregates multiple scale-based distance estimations to ensure that it is more accurate and robust even in situations where the scale changes rapidly (e.g., emergency braking). We demonstrated the proposed system in three different scenes. In the first scene, we were following a truck on the highway, which is the most common situation in truck platooning. We can see the bounding box of the truck and its distance from the ego vehicle. The estimated relative velocity and acceleration are also displayed in the speedometers below. Although the front truck was slightly tilted when approaching a curve, we can see that the bounding box is pretty stable and the distance is estimated accurately. In the second scene, a parked truck that was more than 100m away was approached at about 10m by sudden acceleration and braking. As we can see, the distance, velocity, and acceleration can be estimated stably without losing track even in a situation where relative velocity and acceleration are quite high. In the last scene, both the front vehicle and the ego-vehicle were suffering intensive vibration and the slope was changing in the middle of the scene. Even in this challenging situation, we can see that the front vehicle can be stably tracked, and accurate distance estimation can be achieved. Visit our project page for more information. Project page: https://ishikawa-vision.org/fusion/InterVehicleDistance/index-e.html </video> <video> Robot System for Manipulating a Randomly Placed Towel-like Object Description: In recent years, there has been an increasing demand for robotic handling of towel-like objects in linen and laundry services. One such task involves picking up a towel randomly and aligning it to a predetermined position and posture. However, it is difficult for robots to handle these towel-like flexible objects because deformation occurs during robotic manipulation, and manipulation based on estimation of deformation status is rather challenging and unfeasible for most robotic systems. To solve this problem, in this research, we propose a method to continuously realize the object picking, the object spreading and the object alignment by robot arm system, aiming at further automation. Web Page: http://ishikawa-vision.org/fusion/TowelManipulation/index-e.html </video> <video> Dynamic Perceptive Compensation for Optical Illusions Synchronized with Eye Movement Description: Our dynamic perceptive compensation system realizes controlling illusory perception depending on the eye movement. The system detects user’s eye position by an eye tracker and determine how much compensation is needed for each illusory image based on the illusory characteristics; then the perception is compensated by temporarily changing the displayed image. From the psychophysical experiments, we were able to determine the compensation parameters and observe the temporal dependence of the illusory perception. The technology of controlling gaze-dependent illusions is not limited to understanding the dynamics of visual perception, but enables us to eliminate the visual perception discrepancies that can be applied in the engineering field. *You can partially experience our system compensation in the movie. http://ishikawa-vision.org/perception/RotatingSnakes/index-e.html Result 1: https://doi.org/10.36463/idw.2019.1652 ・(Result 1) Yuki Kubota, Tomohiko Hayakawa, Masatoshi Ishikawa: Reduction of Moving Optical Illusion through Synchronization with Eye Movement, International Display Workshops 2019 (IDW ’19) (Sapporo, 2019.11.27) / Proceedings, INP1-5L (2019). Result 2: https://dl.acm.org/doi/abs/10.1145/3379156.3391344 ・(Result 2) Yuki Kubota, Tomohiko Hayakawa, Masatoshi Ishikawa: Quantitative Perception Measurement of the Rotating Snakes Illusion Considering Temporal Dependence and Gaze Information, Symposium on Eye Tracking Research and Applications (ETRA ’20 Short Papers) (online, 2020.5.27) / Proceedings, No.45, pp. 1-4 (2020). </video> <video> Proximity sensor-based High-speed Tracking and High-precision Depth Scanning Description: We developed one board, USB power supply type, high-speed, high-precision proximity sensor. The sensor element, amplifier circuit, and communication circuit are mounted on a single board (CMOS custom chip and FPGA). The sensor can measure distance and two tilt angles within 1ms. The sensor is suitable for robot application such as control of fingertip position precisely at high speed. And the sensor enables high-precision depth scanning of a target surface. This work was conducted by a joint research project between Ishikawa Group Laboratory and Toyoda Gosei co., ltd. URL: http://ishikawa-vision.org/fusion/prox2/index-e.html </video> <video> 1000分の1秒の世界が新時代を切り拓く Description: 説明 English Version: https://www.youtube.com/watch?v=CO0ChaAccPM 我々の研究室は、2020年4月に、東京大学情報理工学系研究科から情報基盤センターに移りました．このビデオは、東京大学情報理工学系研究科で最後となった2019年度に、それまでの研究の歴史と最近の進歩、並びに研究に対する基本的な姿勢や考え方をまとめたものです． 尚、2020年4月以降の所属は、東京大学情報基盤センター石川グループ研究室 (特任教授 石川正俊) となり、引き続き高速ビジョンとその幅広い応用システムに関する研究を行っております． 研究室Webページ: http: http://ishikawa-vision.org/index-j.html </video> <video> Tracking Projection Mosaicing for Wide High-resolution Display Description: We have developed a tracking projection mosaicing system to wide and high-resolution projection with proper geometric alignment. Considering our gaze, static projectors suffer from a trade-off between an angle of projection and resolution due to limited pixels. Wide-area and high-resolution display is theoretically possible by performing active projection that follows our gaze position. We have focused on tracking projection for laser pointing as our gaze, and realized dynamic high-resolution projection by high-speed visual feedback and appropriate synchronization between a high-speed projector and rotational mirrors. The synchronization enables mosaicing of projected images without any misalignment while dynamically controlling the projector optical axis using the rotational mirrors. Tracking projection mosaicing has advantages in displaying high-resolution photographs, maps, and text over a wide area. The system enables both overall understanding of the wide area and detailed observation of local areas. In future, we will integrate high-speed eye-tracking technology to this projection system for highly immersive display. http://ishikawa-vision.org/mvf/TrackingProjectionMosaicing/index-e.html </video> <video> ElaMorph Projection: Deformation of 3D Shape by Dynamic Projection Mapping Description: We propose a method of illusion named ``ElaMorph projection", in which a rigid object appear to be deformed by dynamic projection mapping. In this example, a plaster of human head appears to have elasticity. The proposed method enables deforming the geometry and rendering in less than 2 ms; this is so fast that human cannot notice the delay of the system. In addition, we refined conventional rendering algorithm which based on the principle of human perception to present effective illusions for 3D solid. Moreover, since environmental lighting is estimated in real time, it is possible to keep the illusion where lighting changes over time, such as party or concert halls. ElaMorph projection expands the range of entertainment of projection mapping. Abstract: http://ishikawa-vision.org/vision/ElaMorph/ Kentaro Fukamizu, Leo Miyashita, and Masatoshi Ishikawa: ElaMorph Projection: Deformation of 3D Shape by Dynamic Projection Mapping, Int. Symposium on Augmented and Mixed Reality (ISMAR 2020) (2020.11.9-13) </video> <video> Motion-blur Compensation Method Using a Transparent Cube Description: Motion blur during imaging in high-speed moving environments greatly reduces the efficiency of inspections and other operations. This study proposes a compensation method using a transparent rotating cube using Snell's law, which can be used even in a scene with large motion blur and can shoot at a high sampling rate. By synchronizing the rotation speed and the moving speed of the object, it is possible not only to capture images without causing motion blur but also to reset the imaging angle of view every 90 ° rotation by taking advantage of the characteristics of the cube. It is characterized by the fact that it requires a smaller illumination device than the conventional optical-axis control method using a Galvano mirror and can capture images at a higher sampling rate. Web URL: https://ishikawa-vision.org/perception/CubeCompensation/index-e.html </video> <video> Fully Automated Beads Art Assembly based on Dynamic Compensation Approach Description: Towards the goal of realizing next-generation manufacturing, we demonstrate the implementation of made-to-order bead art as a simplified scenario of smart manufacturing. Specifically, by configuring a 3DoF position compensation module which is controlled with 1,000Hz visual feedback at the end-effector of an industrial robot arm, the proposed system accurately picks up moving beads on a rotating stage that resembles a belt conveyor and creates bead art according to the design of online orders. The robot achieves good positioning accuracy even under great uncertainties of the moving targets thanks to the dynamic compensation framework, where the high-bandwidth dynamics as well as low-bandwidth dynamics of targets can be accommodated by the compensation module working in a local manner with high-speed visual feedback control and by the robot arm working in a global manner coarsely, respectively. Therefore, we are able to achieve accurate and smart robotic assembly in an unstructured work environment without human intervention. https://ishikawa-vision.org/fusion/BeadsArtAssembly/index-e.html </video> <video> High-resolution Focused Tracking of Freely Swimming Fish Description: We have developed a continuous high-resolution focused imaging system for freely swimming fish using high-speed image processing and optical control. Two high-speed vision system including rotational mirrors track medaka fish continuously, and high-speed liquid lens control by triangulation achieves high-resolution focused imaging for one target. High-speed tracking using ellipse self-windowing can be robust against individual intersection such as tails, and improve efficiency of the tracking. Triangulation with the stereo vision system controls the high-speed liquid lens, and realizes continuous high-resolution focused imaging beyond fixed focus imaging. We can observe in detail fish body textures and the movement of gills and fins even during the swimming. Such observation will lead to individual identification and health management for multiple fish in aquaculture and aquarium, and we also expect this imaging system to have further applications for observation of other dynamic animals or their dynamic specific parts including humans. http://ishikawa-vision.org/mvf/FishTracking/index-e.html </video>

Contacts and Links

Description

Ishikawa Group Laboratory is researching on
1) Sensor Fusion: high speed robot and visual feedback,
2) Dynamic Vision System: 3D image tracking and adaptive optics,
3) System Vision Design: high speed vision and applications, and
4) Active Perception: interactive human interface and media control.

Research Institute for Science and Technology,
Tokyo University of Science

Second YouTube Channel of our Lab.: https://www.youtube.com/IshikawaLab2

Former name: Ishikawa Senoo Lab., Ishikawa Watanabe Lab., Ishikawa Oku Lab., and Ishikawa Komuro Lab.
Japanese name: 石川グループ研究室

Videos from channel