Visualization of work/home density and get a good heat map

-

  • Show the distribution of jobs and homes
  • Technical discussion on how to apply the heatmap properly
  • R code

(Revise on 4/1, on the Friday meeting, a colleague (Xinyue) reminded me a better way to handle the heat map. By taking the log of a highly skewed value the heatmap looks much better)

The distribution of homes and jobs in Chicago

It is an interesting idea to map both home and job density on the same map. The job locations are more concentrated, especially so in the city center.
The 2km x 2km block with most jobs is 13 times the height of the block with most homes. (474 k vs. 37 k).
The 2km x 2km block with most jobs contains 13% of the total jobs, while the one with most homes contains only 1% of all the homes.

Distribution of Jobs:

if use heatmap :

of homes:

if use heatmap:


In animation:




Technical discussion on heatmap:

The jobs are more concentrated in the city center than homes. When the distribution is highly skewed the heatmap cannot show the gradient with enough distinguishability.

In R, ggplot2 has a function cut_number that makes n groups with (approximately) equal numbers of observations. Thus, if n = 4, the intervals are actually Q1, median and Q3.
Work and home density by groups of equal number, the work (top) map should be more concentrated.

 Another function cut_interval makes n groups with equal range. (e.g. [1,4], [4,8], ...)
Neither of them is ideal...
A better solution is to take log of the results, and align the color scale for the two figures:

The scale is based on the number of jobs, which has higher largest value. RColorBrewer is a nice package that can output color scale from the color brewer.


If we don't take log such comparison won't be possible.

The R code:

## take log and cut the variable into groups
breaks_of_interval <- seq(0, max(log(whBlock$workCount)),
                          length.out=7)
whBlock$Range_log_work <- cut(log(whBlock$workCount), breaks = breaks_of_interval)
whBlock$Range_log_home <- cut(log(whBlock$homeCount), breaks = breaks_of_interval)

## find the manual color ##
library("RColorBrewer")
display.brewer.pal(n = 8, name = 'OrRd')
color_scale <- brewer.pal(n = 8, name = "OrRd")[3:8]

# where the people live
ggmap(chicago_map) + 
  geom_tile(data = whBlock, aes(x = long, y = lat, fill = Range_log_home), alpha = 0.8)+
  scale_fill_manual(values = color_scale)+
  theme(legend.position = 'bottom',
        axis.title.y = element_blank(), axis.title.x = element_blank())
# where the people work
ggmap(chicago_map) + 
  geom_tile(data = whBlock, aes(x = long, y = lat, fill = Range_log_work), alpha = 0.8)+
  scale_fill_manual(values = color_scale)+
  theme(legend.position = 'bottom',
        axis.title.y = element_blank(), axis.title.x = element_blank())

For the rest of time, calculated some indexes on similarity and housing / job balance in Chicago. 


Comments

Post a Comment

Popular posts from this blog

How to Draw Heatmap with Colorful Dendrogram

-
Image
Heatmap with Dendrogram: Color Branches and Labels This data visualization example include: * Hierarchical clustering, dendrogram, and heat map based on normalized odds ratios * The dendrogram was built separately to give color to dendrogram’s branches/labels based on clusters using  dendextend * Heatmap is made by  heatmap.2  from  gplots  using the built dendrogram * The rows are sorted by means from highest to lowest, it can be done in either the dendrogram or the  heatmap.2 * Use color palettes from  colorspace Data YRBSS (Youth Risk Behavior Surveillance System) survey data from CDC. This example plots the odds ratios of 63 behavioral questions on one question regarding disordered eating. The odds ratios are median centered by column and log2 transformed Version 1: Color both the branches and labels library (curl) # read file from google drive library (gplots) # heatmap.2 library (dendextend) # make and color dendrogram library (colorspace) #
Read more

Power-law distribution (Pareto)& Zipf's Law: connection and how to fit the distribution of global city population

-
Image
- This post counts for 3 days. - In short,  We use the histogram to outline the distribution, use rank-frequency plot (log-log plot) to identify the power-law distribution, and use maximum likelihood estimation to obtain the parameters. -  I will show 1 st , how the power-law function CDF and PDF can be derived from Zipf CDF; and 2 nd , this least square estimation by excel is wrong and how to get the right one. Historical background on the power-law distribution. - Power-law distribution is as common as normal distribution in the real world. In the 19 th century, Italian economist Pareto realized that 20% of the population owns 80% the wealth, which is the famous 80/20 rule. He later named the power-law distribution as Pareto distribution, which is what we studied in the probability course. - In the 20 th century, another main contributor, Zipf made a similar discovery in linguistics about the frequency of words used. - Besides social wealth, many things follow a hig
Read more

SAS’s Best Subset Selection by Mallows's Cp is actually Stepwise?

-
Image
SAS’s Best Subset is actually Stepwise? I answered this question in my next post:  Best subset selection uses the branch and bound algorithm to speed up The original post explaining my own confusion: I suspect that the function  regsubsets  from R library  leaps  does go through all the possible combinations, and it takes hours for 50 variables. Facing a large number of variables SAS just uses stepwise selection even though the code asks for best subset by adding  selection = cp  to the model part of  proc reg . This is my suspicion, I don’t know whether maybe it never scans all the combinations. I tested with the same dataset from the  last post Without cross-validation, I used all the 598 observations to run  regsubsets : # R: nv_max <- 25 # up-limit of number of variables to test fit_s <- regsubsets(Share_Temporary~., mydata4, really.big = T , nbest= 1 , nvmax = nv_max ) As we can see from the figure above, the number of variables with
Read more