3D Sketching for 3D Object Retrieval

Authors

Bo Li, Juefei Yuan, Yuxiang Ye, Yijuan Lu, Chaoyang Zhang, Qi Tian. (Author Information: Click here).

Submitted to a journal.

 
Abstract

We initiate a pioneer study on 3D sketching and build a 3D sketching system which allows users to freely draw 3D sketches in the air and demonstrate its promising potentials in related applications such as collecting 3D sketch data and conducting 3D sketch-based 3D model retrieval.

Based on the 3D sketching system, we collect a 3D sketch dataset, build a 3D sketch-based 3D model retrieval benchmark, and organize this SHREC track on 3D sketch-based 3D shape retrieval based on the benchmark. We propose a novel 3D sketch-based 3D model retrieval algorithm CNN-SBR based on Convolutional Neural Networks (CNNs) and achieve the top first performance in the SHREC track.

We wish that the 3D sketching system, the 3D sketch-based 3D model retrieval benchmark, and the proposed 3D sketchi-based 3D model retrieval algorithm CNN-SBR will further promote sketch-based 3D shape retrieval and its applications.

 
3D Sketching

 
Fig. 1 3D sketching idea: (a) a user is drawing in the air; (b) a Kinect is tracking the positions (x, y, z) of the user's hand(s): in the bottom right figure, red indicates left hand while green is for right hand.

 
Fig. 2 3D sketching system Graphical User Interface (GUI): (a) GUI of the Kinect tracking subsystem; (b) GUI of the retrieval subsystem (basic version).
 

 
3D Sketch-Based 3D Object Retrieval Systems

 
Fig. 3 Our CNN-SBR: (a) architecture; (b) CNN-SBR's Precision-Recall performance evaluation on the SHREC16STB benchmark.
 

Dataset Collection and Benchmark
 

  • Kinect300 (a 3D Sketch Dataset, with 300 3D Sketches)
 

Source Code
 

 
Paper


Bo Li, Juefei Yuan*, Yuxiang Ye, Yijuan Lu+, Chaoyang Zhang, Qi Tian. 3D Sketching for 3D Object Retrieval. Multimedia Tools and Applications, November, 2020, DOI: https://doi.org/10.1007/s11042-020-10033-1.Bibtex