Journal of Multimedia Information System
Korea Multimedia Society
Section A

Measurement uncertainty evaluation in FaroArm-machine using the bootstrap method

Sherzod Horinov1,*, Khurshid Shaymardanov2, Zafar Tadjiyev3
1CAD-CAM-CAE Center, Turin Polytechnic University in Tashkent, Tashkent, Uzbekistan, +998946080025, sh_horinov@yahoo.com
2CAD-CAM-CAE center, Tashkent, Uzbekistan, khurshid.x@gmail.com
3CAD-CAM-CAE center, Tashkent, Uzbekistan, z.m.tadjiyev@gmail.com
*Corresponding Author: Sherzod Horinov, CAD-CAM-CAE Center, Turin Polytechnic University in Tashkent, Tashkent, Uzbekistan, +998946080025, sh_horinov@yahoo.com.

© Copyright 2015 Korea Multimedia Society. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Jul 30, 2015 ; Revised: Sep 20, 2015 ; Accepted: Oct 12, 2015

Published Online: Sep 30, 2015

Abstract

The modern manufacturing systems and technologies produce products that are more accurate day by day. This can be reached mainly by improvement the manufacturing process with at the same time restricting more and more the quality specifications and reducing the uncertainty in part. The main objective an industry becomes to lower the part’s variability, since the less variability – the better is product. One of the part of this task is measuring the object’s uncertainty. The main purpose of this study is to understand the application of bootstrap method for uncertainty evaluation. Bootstrap method is a collection of sample re-use techniques designed to estimate standard errors and confidence intervals. In the case study a surface of an automobile engine block – (Top view side) is measured by Coordinate Measuring Machine (CMM) and analyzed for uncertainty using Geometric Least Squares in complex with bootstrap method. The designed experiment is composed by three similar measurements (the same features in unique reference system), but with different points (5, 10, 20) concentration at each level. Then each cloud of points was independently analyzed by means of non-linear Least Squares, after estimated results have been reported. A MatLAB software tool used to generate new samples using bootstrap function. The results of the designed experiment are summarized and show that the bootstrap method provides the possibility to evaluate the uncertainty without repeating the Coordinate Measuring Machine (CMM) measurements many times, i.e. potentially can reduce the measuring time.

Keywords: Bootstrap methods; prismatic features; positional tolerances; Faro Arm machine; CAM2 Q measurement software

I. INTRODUCTION

In today’s global and competitive market, companies are facing many challenges especially in shortening the time to market, changing regulations, increasing product complexity and diversity, support and maintenance of products, sustainability. As response to these challenges, manufacturing companies are striving for accelerating innovation and developing lean supply and product processes through lifecycle [5].

Matching customer demands mandate reliable control of production systems. Enhancement of product quality standards implies delving into control and inspection methods. Nowadays, most engineers should work with data, which are extracted from CMM, working with mechanical parts. CMMs are versatile instruments used for precision inspection in industry, and their unique properties more than justify substantial investment; nevertheless, uncertainties associated with them are to be reckoned with. A correct statement of measurement uncertainty is nowadays necessary for companies wishing to comply with ISO 9000 standards, which requires an effective measurement management and measurement process control. Estimation of measurement uncertainty entails taking into account the multipurpose characteristics of coordinate measuring systems. A substantial number of sources of uncertainty is to be reckoned with, including machine components, fixtures and probes, strategy for data acquisition and/or sampling, data processing and measurement environment. The analysis of variance frequently referred to by the contraction ANOVA is a statistical technique specially designed to test whether the means of more than two quantitative populations are equal. This method consists of classifying and cross classifying statistical results and testing whether the means of specified classification differ significantly. In this study, applicability of bootstrap method for testing of several means is discussed and this bootstrap method can be treated as an alternative method to ANOVA when the samples are very small.

Statistical inference based on data resampling has drawn a great deal of attention in recent years. The main idea about these resampling methods is not to assume much about the underlying population distribution and instead tries to get the information about the population from the data itself various types of resampling methods. Bootstrap method [1] (Efron, 1979) use the relationship between the sample and resamples drawn from the sample, to approximate the relationship between the population and samples drawn from it. With the bootstrap method [2], the basic sample is treated as the population and a Monte Carlo style procedure is conducted on it. This is done by randomly drawing a large number of resamples of size n from this original sample with replacement..

Prepare your paper in the same style as used in this sample document. These instructions also contain the necessary information for manual editing. Authors should realize that once the accepted paper is submitted for publication in the journal, the print and online version will be almost identical to the paper submitted for publication, except for the pagination, volume, and copyright year information.

A cover of each paper manuscript should include a title, authors’ names (main author and co-authors), author’s organizations, contact information (email address), and the author’s area of expertise. The first page of a main text should only contain title, abstract with a length of about 200 words, and key-words with around four words.

The title of each paper is expected to cover the whole contents of the paper and it should as concise as possible.

II. BOOTSTRAP METHOD

Types of Bootstrap

Using these types of methods are necessary to make some conclusions after work, which can wields to the main result. For this reason, it should divide into two types of bootstrap method:

Parametric Bootstrap: We know that F belongs to a parametric family of distributions and we just estimate its parameters from the sample. We generate samples from F using the estimated parameters.

Non-parametric Bootstrap: We do not know the form of F and we estimate it by F the empirical distribution obtained from the data.

The sampling distribution of the chosen estimator θ. Both bootstrap and traditional parametric inference seek to achieve the same goal using limited information to estimate the sampling distribution of the chosen estimator. The estimate will be used to make inferences about a population parameter θ. The key difference between these inferential approaches is how they obtain this sampling distribution where as traditional parametric inference utilizes a priori assumptions about the shape of θ s distribution, the non-parametric bootstrap is distribution free, which means that it is not dependent on a particular class of distributions. With the bootstrap method, the entire sampling distribution θ is estimated by relying on the fact that the sample’s distribution is good estimate of the population distribution.

III. BOOTSTRAPPING ANALYSIS: NONPARAMETRIC ANALYSIS

The analysis of data without checking assumptions can be done with bootstrapping techniques. Bradley Efron described a computer intensive technique that resamples collected data in order to study the behavior of a distribution of a specific statistic. As a result, inferences are made on populations that are parametric as well as nonparametric. Bootstrapping is a numerical sampling technique where the data sampled are resampled with replacement. This means that you acquire a sample. Place sample values back and then select another sample. In this way, you get sample data from which you can generate summary statistics for each resample. Various descriptive statistics such as mean, median, mode, variance and correlation can be bootstrapped.

With the use of a computer, the student or researcher can create many resamples. Bootstrapping statistics allows the student or researcher to analyze any distribution and make inferences. The sampled data becomes the population and the resampled data are the samples. There are advantages as well as disadvantages when bootstrapping.

Advantages and Disadvantages of bootstrapping:

Advantages include:

  • Verifying assumptions of normality and equality of variances for the population is unnecessary. Inferences are valid even when assumptions are not verified.

  • There is no need to determine the underlying sampling distribution for any population quantity.

  • Interpretations and results are based upon many observations.

Disadvantages include:

  • Powerful computers are necessary.

  • Randomness must be understood.

  • Computers have built-in error.

  • Large sample sizes must be generated.

Inferential Statistics: Confidence Intervals

In any parametric or nonparametric statistics the researcher’s goal, is to infer something about the population.

In order to make inferences about the population, constructing confidence intervals is acceptable. A confidence interval includes a sample statistic or point estimate plus or minus a constant error term. These ranges of values are found by setting the significance level (usually 0.05) for making a decision for the hypothesis about the population; determining the correct sampling distribution; and finding whether the theorized population value is in the confidence interval or not.

The confidence interval takes on the form point estimate ± a critical value (zα/2) times the standard error (SE) of the statistics. For the mean, the confidence interval looks like

Xbar ± z α/2 SE (sample mean), where xbar is the sample mean.

Testing hypothesis using confidence intervals is accomplished by checking whether the theorized population quantity is or is not in a certain confidence interval. If the theorized population value is in the interval, the null hypothesis is accepted and if not the null hypothesis is rejected. This means that the value obtained in the sample is not significantly different from the theorized population value if in the interval and the sample and population are significantly different when the population value is not in the interval. Suppose a researcher wants to find a 95% confidence interval for the mean. The error term is comprised of the standard error of the mean, that is the standard deviation over the square root of the sample size and a critical z for large samples (where population standard deviation is known) and t for small (where population standard deviation is unknown) samples. These z or t values are critical values and are based of the significance level set by a researcher. Alpha is the type I error or the probability of rejecting a true null hypothesis [4]. For a 95% confidence interval alpha is 1-.95 or 0.05. The values for zα//2 or tα/2 are found from z or t tables or in EXCEL with the application of the function “=TINV (probability, degrees of freedom)”. Consider finding a 95% confidence interval for the mean. Calculating such an interval indicates that upon repeated sampling 95% of the samples will contain the population mean. Suppose a researcher wishes to test the hypothesis that the population mean is 80 from a set of grades for 20 students in a certain statistics course. The data were 81 70 79 86 89 65 76 69 71 88 78 79 81 79 73 84 73 81 89 88. The sample mean was 79.02 and the standard deviation was 7.26. In order to test the hypothesis that sample mean is not significantly different from the theorized mean of 80, a 95% confidence interval is constructed. The researcher never knows what the population mean is, only an approximation. The researcher is 95% confident it is between the numbers that define the interval upon repeated sampling. That is the interval is in agreement with the sample the researcher obtained. If the population mean is outside that interval, then the sample mean is significantly different from the population mean. The results of the calculations of the confidence interval for the mean in this example are as follows: The sample mean is 79.02. The confidence interval is comprised of the sample mean plus or minus tα/2 times the standard error of the mean. In this case, the t value based on an alpha of 0.05 and 20-1 degrees of freedom is 2.09 from a statistical table [4] and the standard error of the mean or the standard deviation divided by the square root of the sample size is 1.62. Combining these numbers provides the 95% confidence interval for the mean as 79.02 ± (2.09) (1.62). The confidence interval is 75.62 < μ < 82.42, where μ is the population mean.

At the 5% or 0.05 significance level, the theorized mean of 80 is contained in the interval found and so the null hypothesis is not rejected. There is no significant difference between the sample mean of 79.02 and the population mean of 80.

Figure 1 shows how to use EXCEL to test this hypothesis about the mean.

jmis-2-3-255-g1
Fig. 1. Hypothesis about the mean
Download Original Figure

If there is no known sampling distribution for a particular theorized population quantity such as median or log means an alternative nonparametric confidence interval is found by using the resampled data and applying the 2.5 percentile and 97.5 percentile for the generated distribution In this way a 95% confidence interval is obtained from the lower 2.5 percentile and upper 97.5 percentile. These bootstrapped confidence intervals are used as any other confidence intervals to make inferences about the population. The examples that follow show the application of EXCEL to inferential statistics and bootstrapping analysis. Classical confidence intervals and bootstrapped percentile confidence intervals are presented. Similar conclusions and results are reached. The determination of bootstrapped percentile confidence intervals is necessary for distributions of medians since there is no known sampling distribution.

IV. RESAMPLING METHODS

The term “resampling” has been applied to a variety of techniques for statistical inference, among which stochastic permutation and the bootstrap are the most characteristic. There are at least four major types of resampling methods which include the randomization exact test, cross-validation, jackknife, and bootstrap. Although today they are unified under a common theme, it is important to note that these four techniques were developed by different people at different periods of time for different purposes. Randomization exact test is a test procedure in which data are randomly re-assigned so that an exact p-value is calculated based on the permutated data.

The underlying idea was to use the power of sampling, in a fashion similar to the way it is used in empirical samples from large universes of data, in order to approximate the ideal test based on the complete set of permutations. Therefore, the idea was to gain the benefits of the classical array of methods - though not a parametric test in this case - by the technical device of simulation sampling. It raises a challenge question on whether the approximation would be satisfactory. For cross-validation, it separates the available data into two or more segments, and then tests the model generated in one segment against the data in the other segment(s). The goal is to find out whether the result is replicable or just a matter of random fluctuations. Supporters of resampling have raised a number of reasons to justify the aforementioned techniques: Empirical, Clarity, Distribution, Small sample size, Large sample size, Replications, Assumption, Generalization, Bias and bad data, Accuracy.

IV. APPLICATION AND USING FARO ARM MACHINE

Applying the bootstrap method using with FARO ARM machine, it should be considered the MATLAB software – the Least Square function- (LSPLANE). The term least squares describes a frequently used approach to solving over determined or inexactly specified systems of equations in an approximate sense. Instead of solving the equations exactly, we seek only to minimize the sum of the squares of the residuals. The least squares criterion has important statistical interpretations [3]. If appropriate probabilistic assumptions about underlying error distributions are made, least squares produces what is known as the maximum-likelihood estimate of the parameters. Even if the probabilistic assumptions are not satisfied, years of experience have shown that least squares produces useful results.

Before extracting using an application and making calculation on it, it should be introduce and show the calibration of machine.

FARO ARM Platinum

The Faro Arm is a portable coordinate measuring machine (CMM) that allows manufacturers easy verification of product quality by performing 3D inspections, tool certifications [Figure 6], CAD comparison, dimensional analysis, reverse engineering, and more. The all-new FARO ARM is the most advanced.

With the Scan Arm, the addition of the FARO Laser Line Probe to the Faro Arm [Figures 2-4] adds unparalleled non-contact 3D scanning capabilities for detailed measurement of surface form, making the Scan Arm the perfect combination of a contact and non-contact portable CMM.

jmis-2-3-255-g2
Fig. 2. Bootstrap Distribution
Download Original Figure
jmis-2-3-255-g3
Fig. 3. Installation FARO ARM
Download Original Figure
jmis-2-3-255-g4
Fig. 4. Installation FARO ARM
Download Original Figure

FARO CAM2 Q is the FARO’s all-in-one metrology software for users that are looking for a single, complete solution for probing and scanning tasks, without the need of any additional software. CAM2 Q is ideal for CAD and non-CAD inspection and Geometric Dimensioning and Tolerancing (GD&T).

Designed with inspection in mind, CAM2 Q can be used in combination with a FARO scanning device, such as the FARO Scan Arm, providing state-of-the-art functionalities for CAD-to-Part analysis. The software is perfectly suited for tasks such as the inspection of free and complex forms and even soft or flexible materials that are difficult to inspect with a tactile system.

The Platinum Faro Arm is lighter than most standard portable CMM units and includes extended-use on-board batteries. The Faro portable CMM platform has a universal 3.5″ mounting ring for easy adaptability and the base offers mounting options [Figure-5] that provide a ”measure-where-you make-it” capability.

jmis-2-3-255-g5
Fig. 5. 1FARO installations options
Download Original Figure

Before calibration, we should make an installation of machine:

[Figure 6-8]

jmis-2-3-255-g6
Fig. 6. Installation FARO ARM
Download Original Figure
jmis-2-3-255-g7
Fig. 7. Calibration Process
Download Original Figure
jmis-2-3-255-g8
Fig. 8. Calibration Process
Download Original Figure
jmis-2-3-255-g9
Fig. 9. Calibration Process
Download Original Figure

Measurement of multiple parts can be simplified using Quick Tools programming [4]. Users can develop part programs quickly by recording the steps in the first measurement. They can also add pictures to the measurement steps to provide easy graphical instructions for other operators. FARO’s Arm system enables users to measure with a Faro Arm or FARO Scan Arm over a larger volume, using the FARO Laser Tracker to accurately determine the position of the arm as it is relocated around the measurement volume. The Arm solution allows all arm measurements to be taken in the coordinate system of the part. Users can pre-set the areas where detailed analysis will be needed. The software simplifies the measurement process by guiding operators to those areas where data must be collected.

After that we can use the machine using the technical task: Applying the software CAM2 Q it should be pointed 5 or 10 points, for instance the block of cylinder: using the MatLAB software it should be considered the least square function:

Analysis of using functions:

jmis-2-3-255-g10
Fig. 10. LSPLANE with 5 points in MatLab
Download Original Figure
jmis-2-3-255-g11
Fig. 11. LSPLANE with 10 points in MatLab
Download Original Figure
jmis-2-3-255-g12
Fig. 12. LSPLANE with 10 points in MatLab
Download Original Figure

IV. CONCLUSION

A comparison is presented, based upon an actual case, between a method for CMM measurement uncertainty estimation based upon simulation, and a standard method. Major problems of current simulative methods were related to strong statistical assumptions in input quantities, quality and quantity of the information required for CMM characterization, and in terms of stringent environmental requirements concerning the laboratory where measurements are performed. The approximated method exhibited good performances: substantial agreement with different measurements of both dimensional and geometrical tolerances was obtained, with no undue overestimation of measurement uncertainty. The approximated method exhibited good performances: substantial agreement with different measurements of both dimensional and geometrical tolerances was obtained, with no undue overestimation of measurement uncertainty. Uncertainty evaluation of CMM measurements is made awkward by the sheer complexity of such machines. Substantial efforts were carried out by academia, national and international institutes and organizations, leading to different approaches for uncertainty estimation [6].

These methods cover e.g. empirically approximated methodologies, counseling advice from measurement experts, and numerical simulations, made possible by the dramatic increase in computing power nowadays available.

Acknowledgement

This work was supported by the laboratory of metrology with the center «CAD-CAM-CAE» in Turin Polytechnic University in Tashkent. The authors would like to acknowledge to Mr. Azamatov A.I.-manager of CAD-CAM-CAE center in Turin Polytechnic University in Tashkent for help and support.

REFERENCES

[1].

B. Efron, “Bootstrap methods,”, Vol. 7, pp 1-26, 1979

[2].

Giulio Barbato and Gianfranco Genta - CIRP JMST-Measurement uncertainty assessment of Coordinate Measuring Machines by simulation and planned experimentation..

[3].

A. Weckenmann, H. Eitzert, M. Garmer, and H. Webert, “Functionality – oriented evaluation and sampling strategy in coordinate metrology,” Precision Engineering, Vol. 17, No. 4, pp. 244-252, 1995.

[4].

B. Efron and R.J. Tibshirani, An Introduction to the bootstrap, Chapman and Hall, London, 1993

[5].

Faro Technologies, Inc.-official book

[6].

PLM 2014, IFIP AICT 442, pp.435-444.