blog-V1

U.S. Food and Beverage Imports Since 1999

Mike Polhemus — Fri, 31 May 2019 14:28:12 GMT

‘U.S. consumers demand variety, quality, and convenience in the foods they consume. As Americans have become wealthier and more ethnically diverse, the American food basket reflects a growing share of tropical products, spices, and imported gourmet products. Seasonal and climatic factors drive U.S. imports of popular types of fruits and vegetables and tropical products, such as cocoa and coffee. In addition, a growing share of U.S. imports can be attributed to intra-industry trade, whereby agricultural-processing industries based in the United States carry out certain processing steps offshore and import products at different levels of processing from their subsidiaries in foreign markets.’ – USDA¹

Fitting Distributions to Censored Data

neil@statpoint.com (Dr. Neil Polhemus) — Wed, 27 Mar 2019 18:57:34 GMT

Real world data frequently contain observations that are only partially known. For example, patients may drop out of medical trials before a study is complete. When collecting environmental data, observations may be less than a reporting limit and therefore lead to a result such as "<1.0". Nevertheless, such censored observations contain useful information that must be accounted for when characterizing the distributions of the population from which the data come.

2018's Top 10 Ways to Visualize Your Data

neil@statpoint.com (Dr. Neil Polhemus) — Fri, 28 Dec 2018 21:56:57 GMT

On December 19, I presented a webinar in which I discussed 10 important techniques for visualizing the information contained in commonly collected data. A recording of the webinar, which lasts just over an hour, may be viewed by clicking here. For those of you who prefer to read rather than watch, I will summarize the top ten visualizations here. Note that the list is subjective and nothing is implied by the sequence in which they are presented.

Fitting Nonlinear Regression Models

neil@statpoint.com (Dr. Neil Polhemus) — Fri, 16 Nov 2018 16:28:04 GMT

The simplest statistical relationship between a dependent variable Y and one or more independent or predictor variables X₁, X₂, ... is

Y = B₀ + B₁X₁ + B₂X₂ + ... + e

where e represents a random deviation from the mean relationship represented by the rest of the model. With a single predictor, the model is a straight line. With more than one predictor, the model is a plane or hyperplane. While such models are adequate for representing many relationships (at least over a limited range of the predictors), there are many cases when a more complicated model is required.

In Statgraphics, there are several procedures for fitting nonlinear models. The models that may be fit include:

1. Transformable nonlinear models: models involving a single predictor variable in which transforming Y, X or both results in a linear relationship between the transformed variables.

2. Polynomial models: models involving one or more predictor variables which include higher-order terms such as B_1,1X₁² or B_1,2X₁X₂.

3. Models that are nonlinear in the parameters: models in which the partial derivatives of Y with respect to the predictor variables involve the unknown parameters.

While the first 2 types of models may be fit using linear least squares techniques, the third requires a numerical search procedure.

In this blog, I will show examples of the 3 types of models and give some advice on fitting them using Statgraphics.

Author Interview at JSM 2018 with CRC Press

neil@statpoint.com (Dr. Neil Polhemus) — Fri, 24 Aug 2018 20:39:14 GMT

During the 2018 Joint Statistical Meetings in Vancouver, CRC Press interviewed me about my book entitled Process Capability Analysis: Estimating Quality. The book contains many examples from Statgraphics, including the analysis of non-normal data and the application of capability analysis to multivariate data. You may view that interview by clicking here.

Hexagon Plots for Big Data

neil@statpoint.com (Dr. Neil Polhemus) — Mon, 11 Jun 2018 16:19:44 GMT

One of the most widely used graphical tools for displaying bivariate data is the X-Y scatterplot, which plots symbols at the location of every observation in a data set. For data sets numbering in the hundreds, such plots usually give the viewer a good idea of any relationship that exists between X and Y. However, for data sets numbering in the hundreds of thousands or even millions, such plots are not very helpful. As an example, consider the scatterplot below which shows data for 6,855,029 commercial flights made in the United States during 2008:

The horizontal axis displays the departure delay for each flight in minutes. The vertical axis displays the associated arrival delay. While general patterns may be seen, including a few outliers, the large amount of overplotting makes it difficult to interpret what's happening where the data is most dense. As we'll see, hexagon plots are one good solution to this problem.

Dynamic Data Visualization: Bringing Data to Life

neil@statpoint.com (Dr. Neil Polhemus) — Wed, 30 May 2018 15:55:20 GMT

In Reston, Virginia on May 16-19, 2018, the Interface Foundation of North America launched its inaugural Symposium on Data Science and Statistics bringing together data scientists, statisticians and machine learning experts to share knowledge and establish new collaborations. It was held in honor of Prof. Edward Wegman of George Mason University. During a session on Dynamic Data Visualization, I presented a talk in which I showed several examples of how adding animation to graphs can help make the underlying data come to life. The slides from that presentation are presented here.

Definitive Screening Designs

neil@statpoint.com (Dr. Neil Polhemus) — Mon, 26 Mar 2018 20:53:01 GMT

The use of experimental design for process optimization is a well-established technique that encourages efficient use of experimental resources. It's one of those areas of statistical analysis where the ROI is relatively easy to quantify and can be extremely high. I still remember when Stu Hunter lectured to us in his course on DOE when I was an undergraduate engineering major. He began each lecture with a story about how he'd used what we were about to learn to help some company improve their processes. He had a unique knack for bringing real life into the classroom and encouraged many of us to pursue careers in statistics.

Classification and Regression Trees

neil@statpoint.com (Dr. Neil Polhemus) — Mon, 26 Feb 2018 21:23:13 GMT

The Classification and Regression Trees procedure added to Statgraphics 18 implements a machine-learning process that may be used to predict observations from data. It creates models of 2 forms:

Text Mining the State of the Union

neil@statpoint.com (Dr. Neil Polhemus) — Thu, 01 Feb 2018 15:53:13 GMT

Data analysts like myself who have been trained as natural scientists and engineers tend to think of data solely as numbers. On the other hand, social scientists know that much information can also be extracted from textual data. Visualization of text using devices such as wordclouds can be very informative, particularly when more than one set of text are compared. The new Statgraphics 18 interface to R provides access to the "tm" text mining package, which opens up new avenues for extracting information from text.