Edward Wegman, George Mason University

Data Reduction by Quantization

Massive data sets challenge the limits of both computability and visualization.
It is therefore desirable to compress data sets; approximately 106 to 107
Bytes seems desirable. This can be done by sampling (thinning) or quantization
(binning). Binning essentially maps the original sample space into a new
discrete sample space. It is commonly thought that data is sparse (lumpy) in
high dimensions. Binning therefore consists of identifying clusters and
determining statistical properties within clusters. Our proposal is to
identify statistical properties within bins and replace original data with
statistically equivalent data of a much smaller scale. This talk explores
these ideas.