2. Visualize: Histogram and Related
David Gerbing
Data
Most of the following examples are an analysis of data in the
Employee data set, included with lessR. First
read the Employee data into the data frame d. See the
Read and Write vignette for more details.
##
## >>> Suggestions
## Recommended binary format for data files: feather
## Create with Write(d, "your_file", format="feather")
## More details about your data, Enter: details() for d, or details(name)
##
## Data Types
## ------------------------------------------------------------
## character: Non-numeric data values
## integer: Numeric data values, integers only
## double: Numeric data values with decimal digits
## ------------------------------------------------------------
##
## Variable Missing Unique
## Name Type Values Values Values First and last values
## ------------------------------------------------------------------------------------------
## 1 Name character 37 0 37 Ritchie, Darnell ... Cassinelli, Anastis
## 2 Years integer 36 1 16 7 NA 7 ... 1 2 10
## 3 Gender character 37 0 2 M M W ... W W M
## 4 Dept character 36 1 5 ADMN SALE FINC ... MKTG SALE FINC
## 5 Salary double 37 0 37 63788.26 104494.58 ... 66508.32 67562.36
## 6 JobSat character 35 2 3 med low high ... high low high
## 7 Plan integer 37 0 3 1 1 2 ... 2 2 1
## 8 Pre integer 37 0 27 82 62 90 ... 83 59 80
## 9 Post integer 37 0 22 92 74 86 ... 90 71 87
## ------------------------------------------------------------------------------------------
As an option, also read the table of variable labels. Create the
table formatted as two columns. The first column is the variable name
and the second column is the corresponding variable label. Not all
variables need be entered into the table. The table can be a
csv file or an Excel file.
Read the label file into the l data frame, currently the
only permitted name. The labels will be displayed on both the text and
visualization output. Each displayed label is the variable name
juxtaposed with the corresponding label, as shown in the following
output.
##
## >>> Suggestions
## Recommended binary format for data files: feather
## Create with Write(d, "your_file", format="feather")
## More details about your data, Enter: details() for d, or details(name)
##
## Data Types
## ------------------------------------------------------------
## character: Non-numeric data values
## ------------------------------------------------------------
##
## Variable Missing Unique
## Name Type Values Values Values First and last values
## ------------------------------------------------------------------------------------------
## 1 label character 8 0 8 Time of Company Employment ... Test score on legal issues after instruction
## ------------------------------------------------------------------------------------------
## label
## Years Time of Company Employment
## Gender Man or Woman
## Dept Department Employed
## Salary Annual Salary (USD)
## JobSat Satisfaction with Work Environment
## Plan 1=GoodHealth, 2=GetWell, 3=BestCare
## Pre Test score on legal issues before instruction
## Post Test score on legal issues after instruction
Histogram
One of the most frequently encountered visualizations for continuous
variables is the histogram, which outlines the general shape of the
underlying distribution.
Histogram: Bin similar values into a group, then plot the
frequency of occurrence of the data values in each bin proportional to
the height of the corresponding bar.
A call to a function to create a histogram contains the name of the
continuous variable that contains the plotted values. With the
X() function, that variable name is the first parameter
value passed to the function. In this example, the only
parameter value passed to the function is the variable name. The data
frame is named d, the default value. The following illustrates
the call to X() with a continuous variable named \(x\).
To illustrate, consider the continuous variable Salary in
the Employee data table. Use X() to tabulate and display
the number of employees in each department, here relying upon the
default data frame (table) named d, so the data=
parameter is not needed.
## [Interactive plot from the Plotly R package (Sievert, 2020)]
## >>> Suggestions
## bin_width: set the width of each bin
## bin_start: set the start of the first bin
## bin_end: set the end of the last bin
## X(Salary, form="density") # smoothed curve + histogram
## X(Salary, form="vbs") # Violin/Box/Scatterplot (VBS) plot
##
## --- Salary: Annual Salary (USD) ---
##
## n miss mean sd min mdn max
## 37 0 83795.557 21799.533 56124.970 79547.600 144419.230
##
##
##
## --- Outliers --- from the boxplot: 1
##
## Small Large
## ----- -----
## 144419.2
##
##
## Bin Width: 10000
## Number of Bins: 10
##
## Bin Midpnt Count Prop Cumul.c Cumul.p
## ---------------------------------------------------------
## 50000 > 60000 55000 4 0.11 4 0.11
## 60000 > 70000 65000 8 0.22 12 0.32
## 70000 > 80000 75000 8 0.22 20 0.54
## 80000 > 90000 85000 5 0.14 25 0.68
## 90000 > 100000 95000 3 0.08 28 0.76
## 100000 > 110000 105000 5 0.14 33 0.89
## 110000 > 120000 115000 1 0.03 34 0.92
## 120000 > 130000 125000 1 0.03 35 0.95
## 130000 > 140000 135000 1 0.03 36 0.97
## 140000 > 150000 145000 1 0.03 37 1.00
##
By default, the X() function provides a color theme
according to the current, active theme. The function also provides the
corresponding frequency distribution, summary statistics, the table that
lists the count of each category, from which the histogram is
constructed, as well as an outlier analysis based on Tukey’s outlier
detection rules for boxplots.
Customize the Histogram
Use the parameters bin_start, bin_width,
and bin_end to customize the histogram.
X(Salary, bin_start=35000, bin_width=14000)
Easy to change the color, either by changing the color theme with
style(), or just change the fill color with
fill. Can refer to standard R colors, as shown with
lessR function showColors(), or implicitly
invoke the lessR color palette generating function
getColors(). Each 30 degrees of the color wheel is named,
such as "greens", "rusts", etc, and implements
a sequential color palette.
Use the color parameter to set the border color, here
turned off.
X(Salary, fill="reds", color="transparent")
The default for formatting both axis labels is to round numeric
values of thousands, such as 100000 to 100K. With parameter
axis_fmt, this default of to {"K"} can be
changed. Also can specify {","} to insert commas in large
numbers with a decimal point or {"."} to insert periods, or
{""} to turn off formatting. The value of
{"K"} can also be combined with {","} or
{"."} by forming a vector of values, such as
c("K", ",").
Axis labels can also formatted by adding a prefix to a numeric value
with the parameter axis_pre, such as $ or
€. The value of axis_pre can be multiple
characters, such as for the Brazilian currency, R$.
X(Salary, axis_fmt=",", axis_x_pre="$")
Frequency Polygon and Proportions
The frequency polygon displays the same binned counts as the
histogram, but marks the count of each bin with a point at the bin’s
midpoint and connects the points with line segments.
X(Salary, form="freq_poly")
By default the height of each bar, or point, is the count of the
values in the bin. Set stat="proportion" to plot the
proportion of all the values in each bin instead, so the heights sum to
1. The shape of the distribution is unchanged, but the vertical scale no
longer depends on the sample size, which makes the distributions from
samples of different sizes comparable.
X(Salary, stat="proportion")
The parameter stat applies to the forms that plot a
frequency, the histogram and the frequency polygon. The other forms,
such as the density curve and the VBS plot, do not plot a frequency, so
for them stat is not available.
Set cumulate="on" for a cumulative histogram, in which
the height of each bar is the count of the values in that bin and all
the bins below it. Set cumulate="both" to superimpose the
standard histogram on the cumulative histogram.
X(Salary, cumulate="both")
The frequency table at the console then also lists the cumulative
counts and cumulative proportions.
Density Plot
The histogram portrays a continuous distribution with discrete bins,
with more modern visualizations available that directly display the
estimated underlying smooth curve.
Density plot: A smooth curve that estimates the underlying
continuous distribution.
To create a density plot, specify the value of form as
"density" parameter. The result is the filled density curve
superimposed on the histogram.
X(Salary, form="density")
The kind parameter indicates the type of density curve.
The default is "general", which follows the shape of the
data. Options are "normal" for the normal density curve
with the same mean and standard deviation as the data, and
"both" for both. Plotting both curves compares the
distribution of the data to a normal distribution, here showing that the
distribution of Salary is skewed to the right, with a longer
tail at the high salaries than a normal curve would have.
X(Salary, form="density", kind="both")
With a by variable, the density curves of the groups are
overlaid in one panel, one curve for each level of by in
its own color. With kind="normal", a normal curve is fit to
each group separately, with that group’s own mean and standard
deviation.
X(Salary, by=Gender, form="density", kind="normal")
## Salary by Gender
##
## Gender n na Mean Median SD IQR Min Max
## M 18 0 91147.46 89792.95 23128.44 27772.74 59188.96 144419.2
## W 19 0 76830.60 71356.69 18438.46 15766.48 56124.97 132563.4
##
## For inferential analysis of the mean difference:
## > ttest(Salary ~ Gender)
The display of the groups is interactive, drawn with the
plotly package. When knitting an R Markdown document,
which writes static graphics, assign the result of the function call to
an object and then display it, as in
p <- X(Salary, by=Gender, form="density"); p.
VBS Plot
A more modern version of the density plot combines the violin plot,
boxplot, and scatter plot into a single visualization, here called the
VBS plot.
The VBS plot is built from three layers, each named by a letter:
v: the violin plot, a density curve and its mirror
image, which shows the shape of the distributionb: the boxplot, which shows the median, the quartiles,
and potential outlierss: the strip plot, a scatterplot of the individual data
values, jittered so that nearby values do not overlap
Specify any one layer or any combination of layers with
form, the letters given in any order and in either case.
For example, form="b" plots the boxplot alone, and
form="vs", the violin plot with the individual data values
but without the boxplot.
Each form that includes the strip layer can show the groups of a
by variable, each group in its own color. The VBS forms
also report summary statistics, which are computed from the data values
as read, not from the jittered points, so they are exact and the same on
every run. Assign the result of the function call to an object to save
the statistics, which are returned as out_pivot, a table of
the statistics for each level of the by variable, here
Gender.
r <- X(Salary, by=Gender, form="vbs")
## Gender n Mean Median SD IQR Min Max
## M 18 91147.46 89792.95 23128.44 27772.74 59188.96 144419.23
## W 19 76830.60 71356.69 18438.46 15766.48 56124.97 132563.38
The men have the higher salaries, with a mean of $91,147 and a median
of $89,793, compared with a mean of $76,831 and a median of $71,357 for
the women.
Stratification
to visualize the distribution of a continuous variable, such as
Salary, grouped by a categorical variable, use the function
X() applied to the continuous variable with by
or facet arguments for the categorical variable, such as
Dept. Better differentiation of the distribution of Salary across the
different groups follows from using facet. The example
below shows a scatterplot of Salary for each level of Dept.
X(Salary, facet=Dept, form="strip")
The two parameters occupy different channels, so both apply at once:
by overlays the groups within a panel and
facet places the panels. Each panel of the following
histogram carries one distribution per level of Gender,
drawn partly transparent so that neither hides the other, with all
panels sharing one set of bins.
X(Salary, by=Gender, facet=Dept)
The same composition applies to the density curve and to the
frequency polygon.
X(Salary, by=Gender, facet=Dept, form="density")
Not every form overlays. The violin plot, the boxplot, and their
composite form="vb" summarize a distribution across the
full width of the panel, so one drawn over another is not readable and
there is no layer to carry the groups. Those forms take
facet alone. The forms that include the strip layer, such
as form="vbs", mark the groups on the plotted points.
X(Salary, by=Gender, facet=Dept, form="vbs")
Orientation
As for Chart(), parameter horiz orients the
categories. With horiz=TRUE, the categories run down the
vertical axis, so the bars, boxes, or curves extend horizontally. For
the violin, box, and strip forms, the categories are the levels of the
facet variable, one panel per row by default. Set
horiz to FALSE to place the panels side by
side on a vertical Salary axis, the familiar layout of boxplots compared
across groups.
X(Salary, facet=Dept, form="box", horiz=FALSE)
For the histogram, the categories are the bins, which by default run
along the horizontal axis, as do the categories of a bar chart from
Chart(). Set horiz to TRUE for
bars that extend horizontally from a vertical Salary axis. The frequency
polygon and density curve rotate the same way. The axis labels follow
their variables, so xlab labels the Salary axis in either
orientation.
XY() has no horiz parameter. Both of its variables are
continuous, so there are no categories to orient. To exchange the axes,
exchange x and y.
Full Manual
Use the base R help() function to view the full manual
for X(). Simply enter a question mark followed by the name
of the function.
?Histogram
More
More on Histograms and other visualizations from
lessR and other packages such as
ggplot2 at:
Gerbing, D., R Visualizations: Derive Meaning from Data, CRC
Press, May, 2020, ISBN 978-1138599635.
