SHREC 2012 - Sketch-Based 3D Shape Retrieval

Call For Participation:

SHREC 2012 - Sketch-Based 3D Shape Retrieval

Objective

The objective of this track is to evaluate the performance of different sketch-based 3D model retrieval algorithms using both hand-drawn and standard line drawings sketch queries on a watertight 3D model dataset.

Introduction
 

(a) Hand-drawn sketches

(b) Standard line drawings

(c) 13 relevant 3D watertight models classes

(d) 7 irrelevant 3D watertight models classes
 
Figure 1 Typical 2D sketches and 3D models

Sketch-based 3D model retrieval is to retrieve 3D models using a 2D sketch as input. This scheme is intuitive and convenient for users to search for relevant 3D models and also important for several applications including sketch-based modeling and sketch-based shape recognition. However, most existing 3D model retrieval algorithms target the Query-by-Model framework, that is, using existing 3D models as queries. Much less research work has been done regarding the Query-by-Sketch framework. In addition, until now there is no comprehensive evaluation or comparison for available sketch-based retrieval algorithms. Considering of this, we organize this track to foster this challenging research area by providing a common sketch-based retrieval benchmark and soliciting retrieval results from current state-of-the-art retrieval methods for comparison. We will also provide corresponding evaluation code for computing a set of performance metrics similar to those used in the Query-by-Model retrieval technique.

Task description
The test dataset will be made available on the 7th of February and the results will be due one week after that. Every participant will perform the queries and send us their retrieval results. We will then do the performance assessment. Participants and organizers will write a joint contest report to detail the results. Results of the track will be presented during the 3DOR workshop 2012 in Cagliari, Italy.

Dataset

Evaluation Methodology
To have a comprehensive evaluation of the retrieval algorithm, we employ seven commonly adopted performance metrics in 3D model retrieval technique. They are Precision-Recall curve (PR), Nearest Neighbor (NN), First Tier (FT), Second Tier (ST), E-Measures (E), Discounted Cumulated Gain (DCG) and Average Precision (AP). We also have developed the code to compute them.

Procedure
The following list is a step-by-step description of the activities:

Schedule
January 28 - Call for participation.
February 1 - Few sample models of the test database will be available on line.
February 5 - Please register before this date.
February 7 - Distribution of the whole database. Participants can start the retrieval.
February 13 - Submission of results (dissimilarity matrix) and a one page description of their method(s).
(Extended to February 15)
February 17 - Distribution of relevance judgments and evaluation scores.
February 23 - Track is finished, and results are ready for inclusion in a track report.
February 26 - Camera ready track papers submitted for printing.
May 13 - Eurographics Workshop on 3D Object Retrieval including SHREC'2012.

Organizers
Bo Li, Afzal Godil - National Institute of Standards and Technology
Tobias Schreck - University of Konstanz

Acknowledgement
We would like to thank Sang Min Yoon (Yonsei University, Korea), Maximilian Scherer (TU Darmstadt, Germany), Tobias Schreck (University of Konstanz) and Arjan Kuijper (Fraunhofer IGD) who collected the TU Darmstadt and Fraunhofer IGD sketch data.
We would also like to thank Daniela Giorgi who built the Watertight Shape Benchmark for SHREC 2007.
We would like to thank Snograss and Vanderwart who built the standard line drawings dataset.

References
[1] R. C. Veltkamp and F. B. ter Haar. SHREC 2007 3D Retrieval Contest. Technical Report UU-CS-2007-015, Department of Information and Computing Sciences, Utrecht University, 2007.
[2] S. M. Yoon, M. Scherer, T. Schreck, and A. Kuijper. Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours. In ACM Multimedia, pages 193-200, 2010.
[3] J. G. Snodgrass and M. Vanderwart. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6(2):174-215, 1980.

Please cite the papers:
[1] B. Li, T. Schreck, A. Godil, M. Alexa, T. Boubekeur, B. Bustos, J. Chen, M. Eitz, T. Furuya, K. Hildebrand, S. Huang, H. Johan, A. Kuijper, R. Ohbuchi, R. Richter, J. M. Saavedra, M. Scherer, T. Yanagimachi, G. J. Yoon, S. M. Yoon, In: M. Spagnuolo, M. Bronstein, A. Bronstein, and A. Ferreira (eds.), SHREC'12 Track: Sketch-Based 3D Shape Retrieval, Eurographics Workshop on 3D Object Retrieval 2012 (3DOR 2012), 2012.
[2] B. Li, Y. Lu, A. Godil, T. Schreck, B. Bustos, A. Ferreira, T. Furuya, M.J. Fonseca, H. Johan, T. Matsuda, R. Ohbuchi, P.B. Pascoal, J.M. Saavedra, A comparison of methods for sketch-based 3D shape retrieval, Computer Vision and Image Understanding (2013), doi: http://dx.doi.org/10.1016/j.cviu.2013.11.008.