Object Detection from Mongolian Nomadic Environmental Images

Perenleilkhundev, Gantuya; Batdemberel, Mungunshagai; Battulga, Batnyam; Batsuuri, Suvdaa

doi:10.33851/JMIS.2019.6.4.173

J Multimed Inf Syst 6(4):173-178

eISSN: 2383-7632

DOI: https://doi.org/10.33851/JMIS.2019.6.4.173

Section A

Object Detection from Mongolian Nomadic Environmental Images

Gantuya Perenleilkhundev¹, Mungunshagai Batdemberel¹, Batnyam Battulga¹, Suvdaa Batsuuri¹^,^*

Author Information & Copyright ▼

¹Department of Information and Computer Sciences, National University of Mongolia, Ulaanbaatar, Mongolia, gantuya@seas.num.edu.mn

^*Corresponding Author : B.Suvdaa, SB district NUM 3^rd building 3-225, Tel: 88001013, E-mail: suvdaa@seas.num.edu.mn.

© Copyright 2019 Korea Multimedia Society. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Nov 18, 2019; Revised: Dec 10, 2019; Accepted: Dec 12, 2019

Published Online: Dec 31, 2019

Abstract

Mongolian historical and cultural monuments on settlement areas of stone inscriptions, stone images, rock-drawings, remains of cities, architecture are still telling us their stories. These monuments depict the understanding of the word, philosophical and artistic outlook, beliefs, religion, national art, language, culture and traditions of Mongols [1]. Nowadays computer science, especially computer vision is applying in the other science fields. The main problem is how to apply and which algorithm can detect and classify the objects correctly. In this paper, we propose a method to detect object from Mongolian nomadic environment images. This work proposes a method for object detection that is the combination of the binary operations in the edge detection results. We found out the best method and parameters of state-of-the-art machine learning algorithms. In experimental result, we evaluate our results with 10-fold cross validation and split 66% strategies.

Keywords: Image processing; Rock-drawing image; Objects detection and classification

I. INTRODUCTION

One of the interesting archeological findings of Mongolia is the drawing of various motives carved or drawn on ancient rocks and statues. It has been some time since research into rock art started in Mongolia. As a result of archaeological study conducted in Mongolia, there are over 500 rock art sites identified and that number is increasing every year due to explorations [7]. There are 2 kinds of rock-drawing images Ochre painting and Petroglyphs. The ochre painting found in Mongolia are divided into three groups;

Animals, various signs and symbols
Cross signs
The images of birds, humans, and animals among numerous dots in square or round frames

The petroglyphs thematically classifying the rock artwork left by the ancient people has great significance to understand its meaning this includes:

Animals themes
Livelihood themes (images related to human activities)
Religious beliefs and funeral rite themes
Seals and their impression
Ambiguous figures

Animal, Livelihood/human activity, seal themed images, and other ambiguous figures are presented almost at all known rock art sites while the images of religious beliefs and funeral rites only occasionally occur.

In this work our goal is to detect two kinds of the objects. One is animal’s whole body or part of body detection from the rock image. The other is stamp or owner’s sign detection from the horse image. We introduced a work that horse stamp recognition [21], that work classify the horse stamp images. That work, we used manually cropped images from horse image. Then in this time, we propose a method to detect stamp image from horse image automatically. The animal rock-images are painted with red ochre or created by engraving the rock surface (engraving the whole body of animals). The most common animals depicted are yangir (wild goat), argali (wild sheep), followed by deer, predators (including wolf, fox, etc), pigs, horse and cattle [1]. Fig.1 shows an example of rock drawing images and horse images.

Fig. 1. (a) Rock-1, (b) Rock-2, (c) Horse-1 and (d) Horse-2.

Download Original Figure

II. RELATED WORKS

In general, there is no specific research using any image processing algorithms for Mongolian nomadic field images. Therefore, we reviewed two kinds of research works that one is rock classification using its color and texture [2], [3] and the other one is drawing image classification using neural network algorithms [4-6]. Leena Lepisto [2] proposed a new method with titled Color and Texture based classification of rock images using classifier combination and Geoffrey Mibei proposed an introduction to types and classification of rocks [3].

The drawing images recognition studies are introduced following researches; Recurrent Neural Networks for Drawing Classification [4], A Convolutional Neural Network in Keras Performs Best [5] and Transfer Learning for Image Classification of various dog breeds [6] show the implementation results that ability of current technologies such as deep learning methods can used art field.

Weixing Wang et al. proposed a new method with titled Rock Fracture Image Segmentation Algorithms [12]. S. Mkwelo, et al. proposed a new algorithm that Watershed-based segmentation of rock scenes and Proximity-based classification of watershed regions under uncontrolled lighting conditions for using mining applications [13].

The rock shape and defect detection and recognition studies based on the edge information are introduced following researches. Effective Adaptive Filter Scale Adjustment Edge Detection Method [14], Edge Detection in Noisy Images, Computational Statistics and Data Analysis [17], Image Segmentation Technique Used in Estimation of The Size Distribution of Rock Fragments in Mining [11] and Study and Comparison of Different Edge Detectors for Image Segmentation [10].

III. METHOD

3.1. Framework structure

To classify the objects from natural field images (see Fig.1), we use several classification steps and find out the best combination and parameters practically. Fig. 2 shows our system structure.

Fig. 2. Schema of Object recognition.

Download Original Figure

First, we have to detect object correctly, and then cropped rock-drawing objects and to classify the objects.

After detecting the objects, we loaded different sized images and resized into 20x20 pixels in grayscale value in Matlab program [8], [9]. Then we used PCA algorithm for feature extraction and tested different classification algorithms to distinguish rock-drawing images of deer from argali (wild sheep).

To detect the rock-drawing part from nomadic environmental rock images, we use several image processing steps and find out the best combination and parameters practically. Fig. 3 shows a structure of our proposed methods for object detection.

Fig. 3. Structure of the proposed method.

Download Original Figure

There are some noises in input images, so we used Gaussian smoothing algorithm for removing the noises. We tested 7 types of images by changing kernel size of Gaussian smooth from 3 to 15 and the best results show in the Table 1. The next step is edge detection [13] using 4 (Canny [11], Roberts, Prewitt, Log) types of popular methods and then combination of them which is implemented logical operations (and, or), select best result among all their possibilities.

Table 1. Smoothing results &I&.

Kernel size	Rock-1	Rock-2	Horse-1	Horse-2
3	60	60	95	90
5	100	88	98	91
7	98	85	88	75
9	100	82	78	55
11	90	48	70	45
13	80	40	60	32
15	85	28	40	20

Download Excel Table

3.2. Principle of Edge Detection

Edge detection operator is a mutation in the nature of the image edge to test the edge. There are two main types [10]: one is the first derivative-based edge detection operator to detect image edges by computing the image gradient values, such as Roberts operator, Sobel operator [10], Prewitt operator; the other one is the second derivative-based edge detection operator, by seeking in the second derivative zero-crossing to edge detection, such as LOG operator, Canny operator.

Gradient is a measure of the function changes. And it is also the first order derivative of the image corresponds to two-dimensional function. An image can be seen as a continuous derivative of image intensity of sampling points group. Gradient [9] is a type of two-dimensional equivalent of the first derivative. It can be defined as a vector.

G (x, y) = [G x G y] = [∂ f / ∂ x ∂ f / ∂ y] .

(1)

There are two important properties. First, the vector G (x, y) direction is same as the direction of the maximum rate of change of increasing function f (x, y) (e.g. formula (2)); Second, the gradient amplitude (e.g. formula (3));

| G (x, y) | = G x 2 + G y 2 G x,

(2)

∝ (x, y) = arctan (G x / G y) .

(3)

For digital images, partial derivative of the edge is almost same as differences. The edge often lies on the differential value of the maximum, minimum, or zero.

G x = f [x + 1, y] − f [x, y], G y = f [x, y + 1] − f [x, y] .

(4)

When we calculate the gradient, the same location (x, y) of real partial derivatives is essential in computing space. Gradient approximation is not in the same location using the above formula. The 2x2 first order differential template is used to calculate partial derivatives in x and y direction of the interpolation points [x +1 / 2, y +1 / 2], then G_x and G_y can be expressed as:

G x = [− 1 1 − 1 1], G y = [11 − 1 − 1] .

(5)

After creating 4 images of edge detection, we use logical 8 combination of them simply. And we selected the best result by comparing their ground truth values.

The last step of our system is applied morphological dilating and closing operations for improving the shape clearly.

IV. EXPERIMENTAL RESULTS

4.1. Data Preparation

We collected data 50 sample gray images from each two classes (argali and deer) of the rock drawing images. Fig. 4 shows 5 samples of the two classes and 10 images of horse with stamp.

Fig. 4. Samples of the collected data (upper 10 samples are rock images and lower 10 samples are horse images)

Download Original Figure

We did experiments in the most popular two kinds of rock-drawing images.

4.2. Object Detection

We did experiments in the most popular rock-drawing images by changing several types of algorithms, with their combination and parameters variations. As a result, we got the best results very near to its ground truth results. Table 1 and Table 2 shows the compared results as percentage of number of bounding boxes in the image.

Table 2. Result of the combination of Logic operations (9).

Logical operations	Rock-1	Rock-2	Horse-1	Horse-2
&&\|	15	13	25	15
&\|&	100	100	78	68
\|\|\|	30	30	10	18
&&&	10	0	0	10
\|&&	40	30	12	19
\|\|&	40	30	13	19
\|&\|	20	30	11	20
&\|\|	35	50	30	35

Download Excel Table

Figure 5 shows the results images according to their steps.

Fig. 5. An example of the results of all steps in an image: (a) input image, (b) gray image, (c) smoothed image, (d) result of logic operations, (e) result of bounding box, and (f) object detected bounding box.

Download Original Figure

The bounding boxes show features or parts of the image objects. We estimate the results using comparison of number of correct bounding boxes and the number of total bounding boxes. The correct bounding boxes includes the feature of ground truth objects. Some bounding box do not include any parts of the object, therefore that result is error.

4.2.1. Results of the Smoothing, Edge detection

We compared results of proposed method with ground truth result, computed the number of correct bounding boxes by dividing the total number of detected bounding boxes (in Table 1 and Table 2). Table 1 shows the results of the different kernel size smoothing when the edge combinations are ‘and or and’ (noted &|&).

After Smoothing filter performs with different kernel sizes from 3 to 15, the kernel size of 9 showed the best result. The best result was 100, 82, 78, and 55%, respectively (in the Table 1). But in the horse images kernel size 3 was the best results. Smoothing is necessary to remove small edges in the horse hair edges and rock image’s growing grasses etc.

4.2.2. Logical operitions for Edge detection

Table 2 shows the result of the logical operition combination of 4 types of edge detection methods' results. The best result was for 4 images ‘and or and’ ( & | & ) operations in all images with 100, 100, 78 and 68% correct results, respactively.

4.3. Classification Results

The detected results cropped by coordinates with minimum x, y and maximum x, y among the all detected bounding boxes.

We tested the results of detected objects using classification for 2 kinds of rock animal images argali and deer (top 2 rows in the Fig.1).

In the feature extraction part, we select several features (10, 25, 35, 50, 100 and without features extraction 400 grayscale pixels) using the PCA method. From experimental results, the best feature dimension was 25. Table 3 shows the results of the classification.

Table 3. Result of the classification methods.

	Feature extraction	25
	Methods	Cross Validation 10	Train 66%
1	Naïve Bayes	100	100
1	Naïve Bayes Multinomial	69.44	65.30
2	Logistic	98.61	100
2	Functions SGD	100	100
3	IBK	100	100
3	LWL	100	100
4	Adaboost M1	98.61	100
4	Random Committee	100	100
5	Zero R	69.44	65.30
5	J Rip	99.30	100
6	Decision Stump	98.61	100
	Random Tree	96.5	100
	Random Forest	100	100

Download Excel Table

The best results were Functions SGD, k-NN, Random forest and the worst methods were Naïve Bayes Multinomial and Zero R classifier. We introduced horse stamp recognition work before [21]. Then in this time, we tested classification results only in the rock-drawing objects.

V. CONCLUSION

In this paper, we had done several experiments to detect objects from nomadic field images, in the most popular rock-drawing images and horse images by changing several types of algorithms, with their combination and parameters variations. As a result, the best in each image as follows: in rock images, kernel size is 9 and logical operations combination of edge detection results are ‘and or and’; in horse images, kernel size is 3 and logical operations combination of edge detection results are ‘and or and’.

Also, we had done several experiments to classify the rock drawing images, in the most popular 2 rock-drawing images by changing several types of algorithms, with their combination and parameters variations. Then, we use PCA method for feature extraction.

Main contribution is to detect object or object parts form nomadic environmental images using combination of several edge detection results. Using this method, it is possible to collect big data for object classification and then it is possible to deep learning methods for cultural information generation from the collected images.

In conclusion, the machine learning methods and its parameters are very sensitive from the structure and type of the rock. In future work, we will do multiclass classification among the other types of the nomadic environmental images by detecting objects. It is possible to classify objects by geographical location, historical time and any other traditional and cultural viewpoints.

Acknowledgement

This research was supported by Young researchers’ grants project (no. P2018-3629) funded by National University of Mongolia in 2018.

REFERENCES

[1].

L. Dashnyam, A. Ochir, N. Urtnasan and D. Tseveendorj, Historical and cultural monuments of Mongolia. Ulaanbaatar, UB: Munkhiin useg Inc., 1999.

[2].

L. Lepisto, “Color and Texture based classification of rock images using classifier combination,” Ph.D thesis, Tampere University of Technology, 2006.

[3].

G. Mibei, “Introduction to types and classification of rocks,” Short Course IX on Exploration for Geothermal Resources, Kenya, vol. 2, no. 24, 2014.

[4].

D. Kradolfer, “Recurrent Neural Networks for Drawing Classification,” https://www.datacareer.ch/blog/quick-draw-classifying-drawings-with-python/, Oct. 2017.

[5].

A. Abdelfattah, “Image Classification using Deep Neural Networks — A beginner friendly approach using TensorFlow,” https://medium.com/@tifa2up/image-classification-using-deep-neural-networks-a-beginner-friendly-approach-using-tensorflow-94b0a090ccd4, Jul 2017.

[6].

P. Devikar, “Transfer Learning for Image Classification of various dog breeds,” International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), vol. 5, no.12, pp. 2707-2715, Dec. 2016.

[7].

N. Batbold, Rock art of Mongolia Archeological Relics of Mongolia. Ulaanbaatar, UB: Munkhiin useg Inc., 2016.

[8].

X. L. Xu, “Application of Matlab in Digital Image Processing, Modern Computer,” Journal of Computer Engineering (IOSRJCE), vol. 2, no. 6, pp. 01-04, Jul. 2012.

[9].

D. F. Zhang, Matlab “Digital Image Processing,” in Proceedings of the 2011 International Conference on Informatics, Cybernetics, and Computer Engineering (ICCE2011), Australia, pp.383-390, Nov. 2011.

[10].

P. P. Acharjya, R. Das and D. Ghoshal, “Study and Comparison of Different Edge Detectors for Image Segmentation,” Global Journal of Computer Science and Technology Graphics & Vision, vol. 12, no. 13, pp. 29-32, Jan. 2012.

[11].

F. Lu, X. Zhou, and Y. He, “Image Segmentation Technique Used in Estimation of The Size Distribution of Rock Fragments in Mining,” IAPR Workshop on CV - Special Hardware and Industrial Applications, Tokyo, Oct. 1988.

[12].

W. Wang, Rock Particle Image Segmentation and Systems, Pattern Recognition Techniques, Technology and Applications, Vienna, Austria. VA: IntechOpen Inc., 2008.

[13].

S. Mkwelo, D. G. Jager, and F. Nicols, “Watershed-based segmentation of rock scenes and proximity-based classification of watershed regions under uncontrolled lighting conditions,” in Proceeding of the 4th annual symposium of the Pattern Recognition Association, pp.107-111, Oct. 2003.

[14].

C. I. Kim, “Adaptive determination of filter scales for edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 5, pp. 579-585, Jun.1992.

[15].

D. Marr and E. Hildreth, “Theory of Edge Detection,” in Proceedings of the Royal Society of London. Series B, Containing papers of a Biological character, Royal Society (Great Britain), London, pp. 187-217, Feb. 1980.

[16].

Q. H. Zhang, S Gao, and T.D Bui, “Edge detection models, Lecture Notes in Computer Science,” in Proceedings of the Royal Society of London. Series B, Biological Sciences, London, pp. 187-217, Dec. 1980.

[17].

D. H. Lim, Robust “Efficient Edge Detection in Noisy Images using Robust Rank-Order Test,” The Korean Journal of Applied Statistics, vol. 20, no. 1, pp. 147-157. Feb. 2007.

[18].

TA. Abbasi, and MU. Abbasi, “A novel FPGA-based architecture for Sobel edge detection operator,” International Journal of Electronics, vol. 94, no. 9, pp. 889-896. Oct. 2007.

[19].

A. C. John, “Computational Approach to Edge Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence,” IEEE transactions on pattern analysis and machine intelligence, vol. 8, no. 6, pp. 679-6987, Nov. 1986.

[20].

Y. Q. Lv and G. Y. Zeng, “Detection Algorithm of Picture Edge,” Journal of Taiyuan Science & Technology, vol.27, no.2, pp. 34-35, Jul. 2009.

[21].

P. Gantuya, B. Mungunshagai, and B. Suvdaa, “Mongolian Traditional Stamp Recognition using Scalable kNN,” International journal of advanced smart convergence, vol. 4, no. 2, pp. 170-176, Dec. 2015.

Authors

Gantuya Perenleilkhundev

jmis-6-4-173-i1

Gantuya Perenleilkhundev received her BS, MS and PhD degrees in Arts in the Department of Industrial design from Mongolian University of Science and Technology, in 1999, 2001 and 2018, respectively. In 2007, she joined the Department of Information and Computer Sciences, School of Engineering and Applied Sciences, National University of Mongolia. Her research interests include horse stamps study, traditional culture and symbols study.

Mungunshagai Batdemberel

jmis-6-4-173-i2

Mungunshagai Batdemberel received his BS and MS degrees in Computer Graphics Design in the Department of Industrial design from Mongolian University of Science and Technology, Mongolia, in 2008. In 2015, he joined the Department of Graphics Design at Ikh Zasag International University of Mongolia. His research interests include 3D modeling and animation, face modeling and computer vision.

Batnyam Battulga

jmis-6-4-173-i3

Batnyam Battulga received his Dipl.-Ing. degrees in Computer Engineer in the Dresden University of Technology in Germany, in 1985-1990 and Dipl.-Inf. Degrees in Software Engineering in the Dresden University of Applied Sciences 1997-2004. He is a senior lecturer at Department of Information and Computer Sciences, at National University of Mongolia. His research interests include software engineering, in special software design, model driven development and agile development.

Suvdaa Batsuuri

jmis-6-4-173-i4

Suvdaa Batsuuri has received her BS and MS degrees in Electronics in the National University of Mongolia, in 2002 and 2004, respectively. In 2011, she received a PhD degree in Computer Science, in the Department of Computer Engineering from Kumoh National Institute of Technology, Korea. From September 2002 to September 2006, she was teacher assistant, laboratory supervisor, assistant lecturer and lecturer at National University of Mongolia. She is an associate professor at Department of Information and Computer Sciences, at National University of Mongolia. Her research interests include image processing, computer vision and machine learning, in special face recognition and distance metric learning. She is a member of IEEE.