Thumbnail
Access Restriction
Subscribed

Author Reinwald, Berthold ♦ Sismanis, Yannis ♦ Gemulla, Rainer ♦ Beyer, Kevin ♦ Haas, Peter J.
Source ACM Digital Library
Content type Text
Publisher Association for Computing Machinery (ACM)
File Format PDF
Language English
Abstract The task of estimating the number of distinct values (DVs) in a large dataset arises in a wide variety of settings in computer science and elsewhere. We provide DV estimation techniques for the case in which the dataset of interest is split into partitions. We create for each partition a synopsis that can be used to estimate the number of DVs in the partition. By combining and extending a number of results in the literature, we obtain both suitable synopses and DV estimators. The synopses can be created in parallel, and can be easily combined to yield synopses and DV estimates for "compound" partitions that are created from the base partitions via arbitrary multiset union, intersection, or difference operations. Our synopses can also handle deletions of individual partition elements. We prove that our DV estimators are unbiased, provide error bounds, and show how to select synopsis sizes in order to achieve a desired estimation accuracy. Experiments and theory indicate that our synopses and estimators lead to lower computational costs and more accurate DV estimates than previous approaches.
Description Affiliation: IBM Almaden Research Center, San Jose, CA. (Beyer, Kevin; Gemulla, Rainer; Haas, Peter J.; Reinwald, Berthold; Sismanis, Yannis)
Age Range 18 to 22 years ♦ above 22 year
Educational Use Research
Education Level UG and PG
Learning Resource Type Article
Publisher Date 2005-08-01
Publisher Place New York
Journal Communications of the ACM (CACM)
Volume Number 52
Issue Number 10
Page Count 9
Starting Page 87
Ending Page 95


Open content in new tab

   Open content in new tab
Source: ACM Digital Library