The Weighted Standard Deviation

-
I have revised the figures in the Conflicts in Colombia paper to add weighted s.e. Together with confidence intervals. Turns out to be not that complicated.
Weighted s.d.:

Add { geom_errorbar(aes(ymin=..., ymax=...), width = 0.2 } in ggplot2 to add the confidence interval into plots:
The paper discussed how conflicts in Colombia makes cities develop more sluggish and denser. I will work more on it in the later posts.
The complete R code for the figure

#   weighted means for global 200 sample
ggplot(data=m4_mean_g_g200, aes(x=year, y=wm, group=Group,
                             color=Group, shape=Group))+
  geom_point(size=4)+
  geom_line(size=1)+
  geom_errorbar(aes(ymin=CI_low, ymax=CI_high), width=.2)+
  ggtitle("Urban Density Change Over Time in 200 Cities") +
  labs(x="Year", y="Average City Density", color = "", shape="") +
  scale_y_continuous(limits = c(30, 150))


Comments

Post a Comment

Popular posts from this blog

Power-law distribution (Pareto)& Zipf's Law: connection and how to fit the distribution of global city population

-
Image
- This post counts for 3 days. - In short,  We use the histogram to outline the distribution, use rank-frequency plot (log-log plot) to identify the power-law distribution, and use maximum likelihood estimation to obtain the parameters. -  I will show 1 st , how the power-law function CDF and PDF can be derived from Zipf CDF; and 2 nd , this least square estimation by excel is wrong and how to get the right one. Historical background on the power-law distribution. - Power-law distribution is as common as normal distribution in the real world. In the 19 th century, Italian economist Pareto realized that 20% of the population owns 80% the wealth, which is the famous 80/20 rule. He later named the power-law distribution as Pareto distribution, which is what we studied in the probability course. - In the 20 th century, another main contributor, Zipf made a similar discovery in linguistics about the frequency of words used. - Besides social wealth, many things fol...
Read more

SAS’s Best Subset Selection by Mallows's Cp is actually Stepwise?

-
Image
SAS’s Best Subset is actually Stepwise? I answered this question in my next post:  Best subset selection uses the branch and bound algorithm to speed up The original post explaining my own confusion: I suspect that the function  regsubsets  from R library  leaps  does go through all the possible combinations, and it takes hours for 50 variables. Facing a large number of variables SAS just uses stepwise selection even though the code asks for best subset by adding  selection = cp  to the model part of  proc reg . This is my suspicion, I don’t know whether maybe it never scans all the combinations. I tested with the same dataset from the  last post Without cross-validation, I used all the 598 observations to run  regsubsets : # R: nv_max <- 25 # up-limit of number of variables to test fit_s <- regsubsets(Share_Temporary~., mydata4, really.big = T , nbest= 1 , nvmax = nv_max ) As we can se...
Read more

Catalog of This Blog and Link to My New Blog

-
Image
When I realized it is so convenient to write blog directly using R Markdown, I searched if there is a specialized tool for it. And I found  Blogdown , an R package written by Yihui Xie, who is also the author of R Markdown. The new blog is hosted on github: https://liuyanguu.github.io/ Catalog As a summary, I would like to create a catalog for the main topics I wrote so far on this blog. 1. Details behind my work at Marron Institute The Zipf’s Law in details and the math behind How to use our city and population weights How I created the city-level GDP per capita How I calculated the distances for airport score 2. Data Visualization The pattern of 3.5 million commuting trips in Chicago Visualization using heatmap 3D World Map: Air Connectivity (Airport Score) Method and New Updates for Airport Score Visualization of Conflicts in Colombia and the World City population, growth rate, and income 3. Introduction to Statistics Boxplot can be vie...
Read more